Skip to content

MicroBatchStream

MicroBatchStream is an extension of the SparkDataStream abstraction for data streams for Micro-Batch Stream Processing.

Contract

Creating PartitionReaderFactory

PartitionReaderFactory createReaderFactory()

PartitionReaderFactory (Spark SQL)

Used when:

Latest Offset

Offset latestOffset()

Latest Offset

Used when:

Input Partitions

InputPartition[] planInputPartitions(
  Offset start,
  Offset end)

InputPartitions (Spark SQL)

Used when:

Implementations