LogicalRDD Leaf Logical Operator¶

LogicalRDD is a leaf logical operator with <> support for a logical representation of a scan over <>.

LogicalRDD is <> when:

• Dataset is requested to checkpoint

• SparkSession is requested to create a DataFrame from an RDD of internal binary rows

=== [[newInstance]] newInstance Method

[source, scala]¶

newInstance(): LogicalRDD.this.type¶

newInstance is part of MultiInstanceRelation abstraction.

newInstance...FIXME

=== [[computeStats]] Computing Statistics -- computeStats Method

[source, scala]¶

computeStats(): Statistics¶

computeStats...FIXME

computeStats is part of the LeafNode abstraction.

Creating Instance¶

LogicalRDD takes the following to be created:

• [[output]] Output schema attributes
• [[rdd]] RDD of InternalRows
• [[outputPartitioning]] Output Partitioning
• [[outputOrdering]] Output ordering (SortOrder)
• [[isStreaming]] isStreaming flag
• [[session]] SparkSession