SparkBatch¶

SparkBatch is a Batch (Spark SQL).

Creating Instance¶

SparkBatch takes the following to be created:

• Snapshot
• Data Schema
• Partition Schema
• Data Schema for Read
• PartitionedFiles
• Predicates
• Data Filters
• Total Bytes
• Read Options
• Hadoop Configuration

SparkBatch is created when:

• SparkScan is requested to toBatch