DeltaOptions¶

DeltaOptions is a DeltaWriteOptions and DeltaReadOptions.

DeltaOptions is a type-safe abstraction of write and read options.

DeltaOptions is used to create WriteIntoDelta command, DeltaSink, and DeltaSource.

DeltaOptions is a Serializable (Java) (so it can be used in Spark tasks).

Creating Instance¶

DeltaOptions takes the following to be created:

• Case-Insensitive Options
• SQLConf (Spark SQL)

When created, DeltaOptions verifies the input options.

DeltaOptions is created when:

• DeltaLog is requested for a relation (for DeltaDataSource as a CreatableRelationProvider and a RelationProvider)
• DeltaCatalog is requested to createDeltaTable
• WriteIntoDeltaBuilder is requested to buildForV1Write
• CreateDeltaTableCommand is executed
• DeltaDataSource is requested for a streaming source (to create a DeltaSource for Structured Streaming), a streaming sink (to create a DeltaSink for Structured Streaming), and for an insertable HadoopFsRelation

Verifying Options¶

verifyOptions(
  options: CaseInsensitiveMap[String]): Unit

verifyOptions finds invalid options among the input options.

verifyOptions is used when:

• DeltaOptions is created
• DeltaDataSource is requested for a relation (for loading data in batch queries)