UnresolvedFunction is an Expression that represents a function (application) in a logical query plan.
UnresolvedFunction is <
callUDF standard function
RelationalGroupedDataset.agg operator with aggregation functions specified by name (that converts function names to UnresolvedFunction expressions)
AstBuilderis requested to sql/AstBuilder.md#visitFunctionCall[visitFunctionCall] (in SQL queries)
UnresolvedFunction can never be Expression.md#resolved[resolved] (and is replaced at analysis phase).
UnresolvedFunction is first looked up in LookupFunctions logical rule and then resolved in ResolveFunctions logical resolution rule.
UnresolvedFunction can never be resolved it should not come as a surprise that it cannot be evaluated either (i.e. produce a value given an internal row). When requested to evaluate,
UnresolvedFunction simply reports a
Cannot evaluate expression: [this]
TIP: Use Catalyst DSL's function or distinctFunction to create a
UnresolvedFunction with <
=== [[apply]] Creating UnresolvedFunction (With Database Undefined) --
apply Factory Method
apply(name: String, children: Seq[Expression], isDistinct: Boolean): UnresolvedFunction¶
apply creates a
FunctionIdentifier with the
name and no database first and then creates a <
apply is used when:
callUDF standard function is used
RelationalGroupedDatasetis requested to agg with aggregation functions specified by name (and converts function names to UnresolvedFunction expressions)
UnresolvedFunction takes the following to be created:
- [[children]] Child Expression.md[expressions]
// Using Catalyst DSL to create UnresolvedFunctions import org.apache.spark.sql.catalyst.dsl.expressions._ // Scala Symbols supported only val f = 'f.function() scala> :type f org.apache.spark.sql.catalyst.analysis.UnresolvedFunction scala> f.isDistinct res0: Boolean = false val g = 'g.distinctFunction() scala> g.isDistinct res1: Boolean = true