Skip to content

UserDefinedFunction

UserDefinedFunction is an abstraction of user-defined functions (UDFs).

Contract (Subset)

Creating Column for Execution

apply(
  exprs: Column*): Column

Column with Expression to execute this UserDefinedFunction

withName

withName(
  name: String): UserDefinedFunction

Associates the given name with this UserDefinedFunction

Used when:

Implementations

Demo

import org.apache.spark.sql.functions.udf
val lengthUDF = udf { s: String => s.length }
scala> :type lengthUDF
org.apache.spark.sql.expressions.UserDefinedFunction
val r = lengthUDF($"name")
scala> :type r
org.apache.spark.sql.Column
val namedLengthUDF = lengthUDF.withName("lengthUDF")
scala> namedLengthUDF($"name")
res2: org.apache.spark.sql.Column = UDF:lengthUDF(name)
val nonNullableLengthUDF = lengthUDF.asNonNullable
assert(nonNullableLengthUDF.nullable == false)