DataSourceScanExec Leaf Physical Operators¶

DataSourceScanExec is an extension of the LeafExecNode abstraction for leaf physical operators that represent scans over a BaseRelation.

DataSourceScanExec uses scan for the variable name prefix for Whole-Stage Java Code Generation.

Contract¶

Input RDDs¶

inputRDDs(): Seq[RDD[InternalRow]]

Metadata¶

metadata: Map[String, String]

BaseRelation¶

relation: BaseRelation

BaseRelation

TableIdentifier¶

tableIdentifier: Option[TableIdentifier]

Implementations¶

• FileSourceScanExec
• RowDataSourceScanExec

Node Name¶

nodeName: String

nodeName is part of the TreeNode abstraction.

nodeName is the following text (with the relation and the tableIdentifier):

Scan [relation] [tableIdentifier]

Simple Node Description¶

simpleString(
  maxFields: Int): String

simpleString is part of the TreeNode abstraction.

simpleString is the following text (with the nodeNamePrefix, the nodeName and the metadata redacted and truncated to spark.sql.maxMetadataStringLength characters):

[nodeNamePrefix][nodeName][comma-separated output][metadata]

Node Name Prefix¶

nodeNamePrefix: String

nodeNamePrefix is an empty text.