Skip to content

LogicalQueryStage Leaf Logical Operator

LogicalQueryStage is a leaf logical operator for Adaptive Query Execution.

Creating Instance

LogicalQueryStage takes the following to be created:

LogicalQueryStage is created when:

Query Optimization

LogicalQueryStage is a "target" of the following logical optimizations:

Query Planning

LogicalQueryStage is planned by LogicalQueryStageStrategy execution planning strategy.

Statistics

computeStats(): Statistics

computeStats tries to find the first QueryStageExec leaf physical operators in the physical plan that is then requested for the statistics.

computeStats prints out the following DEBUG messages to the logs based on the availability of the statistics.

Physical stats available as [physicalStats] for plan: [physicalPlan]
Physical stats not available for plan: [physicalPlan]

In the end, computeStats gives the statistics of the physical operator or requests the logical plan for them.

computeStats is part of the LeafNode abstraction.

Logging

Enable ALL logging level for org.apache.spark.sql.execution.adaptive.LogicalQueryStage logger to see what happens inside.

Add the following line to conf/log4j.properties:

log4j.logger.org.apache.spark.sql.execution.adaptive.LogicalQueryStage=ALL

Refer to Logging.