Skip to content

CreateNamedStruct

CreateNamedStruct is an Expression.

Creating Instance

CreateNamedStruct takes the following to be created:

CreateNamedStruct is created when:

Never Nullable

nullable: Boolean

nullable is always disabled (false).

nullable is part of the Expression abstraction.

Node Patterns

nodePatterns: Seq[TreePattern]

nodePatterns is CREATE_NAMED_STRUCT.

nodePatterns is part of the TreeNode abstraction.

Pretty Name

prettyName: String

prettyName is either function alias for the functionAliasName tag (if defined) or named_struct.

prettyName is part of the Expression abstraction.

Interpreted Expression Evaluation

eval(
  input: InternalRow): Any

eval creates an InternalRow with the result of evaluation of all the value expressions.

eval is part of the Expression abstraction.

Code-Generated Expression Evaluation

doGenCode(
  ctx: CodegenContext,
  ev: ExprCode): ExprCode

doGenCode is part of the Expression abstraction.

import org.apache.spark.sql.functions.lit
val exprs = Seq("a", 1).map(lit).map(_.expr)

import org.apache.spark.sql.catalyst.dsl.expressions._
val ns = namedStruct(exprs: _*)

// doGenCode is used when Expression.genCode is executed

import org.apache.spark.sql.catalyst.expressions.codegen.CodegenContext
val ctx = new CodegenContext
val code = ns.genCode(ctx).code

scala> println(code)
Object[] values_0 = new Object[1];


if (false) {
  values_0[0] = null;
} else {
  values_0[0] = 1;
}

final InternalRow value_0 = new org.apache.spark.sql.catalyst.expressions.GenericInternalRow(values_0);
values_0 = null;

FunctionRegistry

CreateNamedStruct is registered in FunctionRegistry under the name of named_struct SQL function.

import org.apache.spark.sql.catalyst.FunctionIdentifier
val fid = FunctionIdentifier(funcName = "named_struct")
val className = spark.sessionState.functionRegistry.lookupFunction(fid).get.getClassName

scala> println(className)
org.apache.spark.sql.catalyst.expressions.CreateNamedStruct

val q = sql("SELECT named_struct('id', 0)")
// analyzed so the function is resolved already (using FunctionRegistry)
val analyzedPlan = q.queryExecution.analyzed

scala> println(analyzedPlan.numberedTreeString)
00 Project [named_struct(id, 0) AS named_struct(id, 0)#7]
01 +- OneRowRelation

val e = analyzedPlan.expressions.head.children.head
import org.apache.spark.sql.catalyst.expressions.CreateNamedStruct
assert(e.isInstanceOf[CreateNamedStruct])

Catalyst DSL

Catalyst DSL defines namedStruct operator to create a CreateNamedStruct expression.

import org.apache.spark.sql.catalyst.dsl.expressions._
val s = namedStruct()

import org.apache.spark.sql.catalyst.expressions.CreateNamedStruct
assert(s.isInstanceOf[CreateNamedStruct])

val s = namedStruct("*")

scala> println(s)
named_struct(*)