TahoeChangeFileIndex¶
TahoeChangeFileIndex
is a TahoeFileIndexWithSnapshotDescriptor of AddCDCFiles for changesToDF in Change Data Feed.
Creating Instance¶
TahoeChangeFileIndex
takes the following to be created:
-
SparkSession
(Spark SQL) - CDCDataSpecs of AddCDCFile
- DeltaLog
- Path
- SnapshotDescriptor
TahoeChangeFileIndex
is created when:
CDCReaderImpl
is requested to changesToDF
matchingFiles¶
TahoeFileIndex
matchingFiles(
partitionFilters: Seq[Expression],
dataFilters: Seq[Expression]): Seq[AddFile]
matchingFiles
is part of the TahoeFileIndex abstraction.
matchingFiles
...FIXME
inputFiles¶
inputFiles
is the absolute paths of all the AddCDCFiles of the CDCDataSpecs.