Skip to content

DeltaOptions

DeltaOptions is a type-safe abstraction of the supported write and read options.

DeltaOptions is used to create WriteIntoDelta command, DeltaSink, and DeltaSource.

Creating Instance

DeltaOptions takes the following to be created:

  • Case-Insensitive Options
  • SQLConf (Spark SQL)

When created, DeltaOptions verifies the options.

DeltaOptions is created when:

Verifying Options

verifyOptions(
  options: CaseInsensitiveMap[String]): Unit

verifyOptions finds invalid options among the input options.

Note

In the open-source version verifyOptions does really nothing. The underlying objects (recordDeltaEvent and the others) are no-ops.

verifyOptions is used when:

Serializable

DeltaOptions is a Serializable (Java) (so it can be used in Spark tasks).

maxRecordsPerFile

maxRecordsPerFile