StreamingWrite¶

StreamingWrite is an abstraction of streaming writers.

Contract¶

Aborting Writing Job¶

void abort(
  long epochId,
  WriterCommitMessage[] messages)

Used when:

• MicroBatchWrite is requested to abort

Committing Writing Job¶

void commit(
  long epochId,
  WriterCommitMessage[] messages)

Used when:

• EpochCoordinator is requested to commitEpoch
• MicroBatchWrite is requested to commit

Creating StreamingDataWriterFactory¶

StreamingDataWriterFactory createStreamingWriterFactory(
  PhysicalWriteInfo info)

Used when:

• MicroBatchWrite is requested to createBatchWriterFactory
• WriteToContinuousDataSourceExec physical operator is requested to execute

Implementations¶

• ForeachWrite
• ConsoleWrite
• KafkaStreamingWrite
• MemoryStreamingWrite
• NoopStreamingWrite