TahoeChangeFileIndex¶
TahoeChangeFileIndex is a TahoeFileIndexWithSnapshotDescriptor of AddCDCFiles for changesToDF in Change Data Feed.
Creating Instance¶
TahoeChangeFileIndex takes the following to be created:
-
SparkSession(Spark SQL) - CDCDataSpecs of AddCDCFile
- DeltaLog
- Path
- SnapshotDescriptor
TahoeChangeFileIndex is created when:
CDCReaderImplis requested to changesToDF
matchingFiles¶
TahoeFileIndex
matchingFiles(
partitionFilters: Seq[Expression],
dataFilters: Seq[Expression]): Seq[AddFile]
matchingFiles is part of the TahoeFileIndex abstraction.
matchingFiles...FIXME
inputFiles¶
inputFiles is the absolute paths of all the AddCDCFiles of the CDCDataSpecs.