TypedColumn¶

TypedColumn is a Column with the ExpressionEncoder for the types of the input and the output.

TypedColumn is created using as operator on a Column.

scala> val id = $"id".as[Int]
id: org.apache.spark.sql.TypedColumn[Any,Int] = id

scala> id.expr
res1: org.apache.spark.sql.catalyst.expressions.Expression = 'id

=== [[name]] name Operator

name(
  alias: String): TypedColumn[T, U]

name is part of the Column abstraction.

name...FIXME

=== [[withInputType]] Creating TypedColumn -- withInputType Internal Method

[source, scala]¶

withInputType( inputEncoder: ExpressionEncoder[_], inputAttributes: Seq[Attribute]): TypedColumn[T, U]

withInputType...FIXME

withInputType is used when the following typed operators are used:

• Dataset.select

• KeyValueGroupedDataset.agg

• RelationalGroupedDataset.agg

Creating Instance¶

TypedColumn takes the following when created:

• [[expr]] Catalyst expressions/Expression.md[expression]
• [[encoder]] ExpressionEncoder of the column results

TypedColumn initializes the <>.