DeltaOptions is a type-safe abstraction of the supported write and read options.
DeltaOptions is used to create WriteIntoDelta command, DeltaSink, and DeltaSource.
DeltaOptions takes the following to be created:
- Case-Insensitive Options
DeltaOptions verifies the options.
DeltaOptions is created when:
DeltaLogis requested for a relation (for DeltaDataSource as a CreatableRelationProvider and a RelationProvider)
DeltaCatalogis requested to createDeltaTable
WriteIntoDeltaBuilderis requested to buildForV1Write
- CreateDeltaTableCommand is executed
DeltaDataSourceis requested for a streaming source (to create a DeltaSource for Structured Streaming), a streaming sink (to create a DeltaSink for Structured Streaming), and for an insertable HadoopFsRelation
verifyOptions( options: CaseInsensitiveMap[String]): Unit
verifyOptions finds invalid options among the input
In the open-source version
verifyOptions does really nothing. The underlying objects (
recordDeltaEvent and the others) are no-ops.
verifyOptions is used when:
DeltaDataSourceis requested for a relation (for loading data in batch queries)
DeltaOptions is a
Serializable (Java) (so it can be used in Spark tasks).