ScalarSubquery (ExecSubqueryExpression)¶

ScalarSubquery is an ExecSubqueryExpression that <> (i.e. the value of executing <> subquery that can result in a single row and a single column or null if no row were computed).

IMPORTANT: Spark SQL uses the name of ScalarSubquery twice to represent an ExecSubqueryExpression (this page) and a SubqueryExpression. It is confusing and you should not be anymore.

ScalarSubquery is <> when PlanSubqueries physical optimization is executed (and plans a ScalarSubquery expression).

[source, scala]¶

// FIXME DEMO import org.apache.spark.sql.execution.PlanSubqueries val spark = ... val planSubqueries = PlanSubqueries(spark) val plan = ... val executedPlan = planSubqueries(plan)

[[Unevaluable]] ScalarSubquery is an unevaluable expression.

[[dataType]] ScalarSubquery uses...FIXME...for the <>.

[[internal-registries]] .ScalarSubquery's Internal Properties (e.g. Registries, Counters and Flags) [cols="1,2",options="header",width="100%"] |=== | Name | Description

| result | [[result]] The value of the single column in a single row after collecting the rows from executing the <> or null if no rows were collected.

| updated | [[updated]] Flag that says whether ScalarSubquery was <> with collected result of executing the <>. |===

=== [[creating-instance]] Creating ScalarSubquery Instance

ScalarSubquery takes the following when created:

• [[plan]] SubqueryExec.md[SubqueryExec] plan
• [[exprId]] Expression ID (as ExprId)

=== [[updateResult]] Updating ScalarSubquery With Collected Result -- updateResult Method

[source, scala]¶

updateResult(): Unit¶

NOTE: updateResult is part of spark-sql-Expression-ExecSubqueryExpression.md#updateResult[ExecSubqueryExpression Contract] to fill an Catalyst expression with a collected result from executing a subquery plan.

updateResult requests <> physical plan to SubqueryExec.md#executeCollect[execute and collect internal rows].

updateResult sets <> to the value of the only column of the single row or null if no row were collected.

In the end, updateResult marks the ScalarSubquery instance as <>.

updateResult reports a RuntimeException when there are more than 1 rows in the result.

more than one row returned by a subquery used as an expression:
[plan]

updateResult reports an AssertionError when the number of fields is not exactly 1.

Expects 1 field, but got [numFields] something went wrong in analysis

=== [[eval]] Evaluating Expression -- eval Method

[source, scala]¶

eval(input: InternalRow): Any¶

eval is part of the Expression abstraction.

eval simply returns <> value.

eval reports an IllegalArgumentException if the ScalarSubquery expression has not been <> yet.

=== [[doGenCode]] Generating Java Source Code (ExprCode) For Code-Generated Expression Evaluation -- doGenCode Method

[source, scala]¶

doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode¶

NOTE: doGenCode is part of <> to generate a Java source code (ExprCode) for code-generated expression evaluation.

doGenCode first makes sure that the <> flag is on (true). If not, doGenCode throws an IllegalArgumentException exception with the following message:

requirement failed: [this] has not finished

doGenCode then creates a <> (for the <> and the <>) and simply requests it to <>.