DeltaCDFRelation¶

DeltaCDFRelation is a BaseRelation (Spark SQL) and a PrunedFilteredScan (Spark SQL).

Creating Instance¶

DeltaCDFRelation takes the following to be created:

• SnapshotWithSchemaMode
• SQLContext (Spark SQL)
• Optional start version
• Optional end version

DeltaCDFRelation is created when:

• CDCReaderImpl is requested for a CDF-aware BaseRelation

buildScan¶

buildScan(
  requiredColumns: Array[String],
  filters: Array[Filter]): RDD[Row]

buildScan changesToBatchDF.

In the end, buildScan selects the given requiredColumns (using Dataset.select operator) and requests it for the RDD[Row].