BatchScanExec Physical Operator¶

BatchScanExec is a DataSourceV2ScanExecBase leaf physical operator for scanning a batch of data from a Scan.

BatchScanExec represents a data scan over a DataSourceV2ScanRelation relation at execution.

Creating Instance¶

BatchScanExec takes the following to be created:

• Output Schema (Seq[AttributeReference])
• Scan
• Runtime Filters (Expressions)
• Optional SortOrders
• Table
• StoragePartitionJoinParams

BatchScanExec is created when:

• DataSourceV2Strategy execution planning strategy is executed (for physical operators with a DataSourceV2ScanRelation relation)

Input RDD¶

inputRDD: RDD[InternalRow]

For no filteredPartitions and the outputPartitioning to be SinglePartition, inputRDD creates an empty RDD[InternalRow] with 1 partition.

Otherwise, inputRDD creates a DataSourceRDD as follows:

Filtered Input Partitions¶

filteredPartitions: Seq[Seq[InputPartition]]

For non-empty runtimeFilters, filteredPartitions...FIXME

Otherwise, filteredPartitions is the partitions (that usually is the input partitions of this BatchScanExec).

Input Partitions¶

inputPartitions: Seq[InputPartition]

inputPartitions requests the Batch to plan input partitions.

PartitionReaderFactory¶

readerFactory: PartitionReaderFactory

readerFactory requests the Batch to createReaderFactory.

Batch¶

batch: Batch

batch requests the Scan for the physical representation for batch query.

batch is used when:

• BatchScanExec is requested for partitions and readerFactory

keyGroupedPartitioning¶

keyGroupedPartitioning: Option[Seq[Expression]]

keyGroupedPartitioning requests this StoragePartitionJoinParams for the keyGroupedPartitioning.