UserDefinedFunction¶
UserDefinedFunction is an abstraction of user-defined functions (UDFs).
Contract (Subset)¶
Creating Column (for Function Execution)¶
apply(
exprs: Column*): Column
Column with Expression to execute this UserDefinedFunction
withName¶
withName(
name: String): UserDefinedFunction
Associates the given name with this UserDefinedFunction
Used when:
UDFRegistrationis requested to register a named UserDefinedFunction
Implementations¶
Demo¶
import org.apache.spark.sql.functions.udf
val lengthUDF = udf { s: String => s.length }
scala> :type lengthUDF
org.apache.spark.sql.expressions.UserDefinedFunction
val r = lengthUDF($"name")
scala> :type r
org.apache.spark.sql.Column
val namedLengthUDF = lengthUDF.withName("lengthUDF")
scala> namedLengthUDF($"name")
res2: org.apache.spark.sql.Column = UDF:lengthUDF(name)
val nonNullableLengthUDF = lengthUDF.asNonNullable
assert(nonNullableLengthUDF.nullable == false)