BatchWrite¶

BatchWrite is an abstraction of writers to a batch data source.

Contract¶

Aborting Write Job¶

void abort(
  WriterCommitMessage[] messages)

Used when:

• V2TableWriteExec physical command is requested to writeWithV2

Committing Write Job¶

void commit(
  WriterCommitMessage[] messages)

Used when:

• V2TableWriteExec physical command is requested to writeWithV2

Creating Batch DataWriterFactory¶

DataWriterFactory createBatchWriterFactory(
  PhysicalWriteInfo info)

Used when:

• V2TableWriteExec physical command is requested to writeWithV2

onDataWriterCommit¶

void onDataWriterCommit(
  WriterCommitMessage message)

onDataWriterCommit does nothing by default (noop).

Used when:

• V2TableWriteExec physical command is requested to writeWithV2

useCommitCoordinator¶

boolean useCommitCoordinator()

Controls whether this writer requires a Commit Coordinator to coordinate writing tasks (and ensure that at most one task for each partition commits).

Default: true

Used when:

• V2TableWriteExec physical command is requested to writeWithV2

Implementations¶

• FileBatchWrite
• KafkaBatchWrite
• MicroBatchWrite (Spark Structured Streaming)
• NoopBatchWrite