SupportsRealTimeMode Streaming Sources¶

SupportsRealTimeMode is an abstraction of streaming sources that can be used in Real-Time Mode.

Contract¶

mergeOffsets¶

Offset mergeOffsets(
  PartitionOffset[] offsets)

See:

• KafkaMicroBatchStream
• LowLatencyMemoryStream

Used when:

• MicroBatchExecution stream execution engine is requested to markMicroBatchEnd

planInputPartitions¶

InputPartition[] planInputPartitions(
  Offset start)

InputPartitions for the start offset

See:

• KafkaMicroBatchStream
• LowLatencyMemoryStream

Used when:

• RealTimeStreamScanExec physical operator is requested for the inputPartitions

prepareForRealTimeMode¶

void prepareForRealTimeMode()

Informs this source that it is executed in real-time mode (with RealTimeTrigger)

See:

• KafkaMicroBatchStream

Used when:

• MicroBatchExecution stream execution engine is requested for the LogicalPlan

Implementations¶

• KafkaMicroBatchStream
• LowLatencyMemoryStream