Skip to content

SparkDataStream¶

SparkDataStream is an abstraction of readable data streams.

Contract¶

commit¶

void commit(
  Offset end)

Used when:

ContinuousExecution stream execution engine is requested to commit
MicroBatchExecution stream execution engine is requested to constructNextBatch

deserializeOffset¶

Offset deserializeOffset(
  String json)

Used when:

ContinuousExecution stream execution engine is requested to runContinuous and commit
MicroBatchExecution stream execution engine is requested to constructNextBatch and runBatch

initialOffset¶

Offset initialOffset()

Used when:

ContinuousExecution stream execution engine is requested to runContinuous
MicroBatchExecution stream execution engine is requested to constructNextBatch and runBatch

stop¶

void stop()

Used when:

StreamExecution is requested to stop sources

Implementations¶