Skip to content

RatePerMicroBatchStream

RatePerMicroBatchStream is a MicroBatchStream that SupportsTriggerAvailableNow.

Creating Instance

RatePerMicroBatchStream takes the following to be created:

RatePerMicroBatchStream is created when:

isTriggerAvailableNow Flag

RatePerMicroBatchStream defines isTriggerAvailableNow flag to determine whether it is executed in Trigger.AvailableNow mode or not (based on prepareForTriggerAvailableNow).

By default, isTriggerAvailableNow flag is off (false).

isTriggerAvailableNow is used when:

prepareForTriggerAvailableNow

prepareForTriggerAvailableNow(): Unit

prepareForTriggerAvailableNow is part of the SupportsTriggerAvailableNow abstraction.


prepareForTriggerAvailableNow turns the isTriggerAvailableNow flag on (for Trigger.AvailableNow mode).

Latest Offset

latestOffset(
  startOffset: Offset,
  limit: ReadLimit): Offset

latestOffset is part of the SupportsAdmissionControl abstraction.


latestOffset is different whether it is executed in Trigger.AvailableNow mode or not (based on isTriggerAvailableNow flag).

latestOffset calculates next offset just once for Trigger.AvailableNow and the offsetForTriggerAvailableNow registry has no value assigned yet.

In other words, for isTriggerAvailableNow, latestOffset returns the value of the offsetForTriggerAvailableNow registry (that remains the same for every latestOffset).

For all the other triggers (when isTriggerAvailableNow is disabled), latestOffset calculates next offset.

Calculating Next Offset

calculateNextOffset(
  start: Offset): Offset

calculateNextOffset extractOffsetAndTimestamp from the given start Offset.

calculateNextOffset creates a RatePerMicroBatchStreamOffset for the end offset (that is rowsPerBatch rows long):

  • Increments the start offset by the rowsPerBatch for the end offset
  • Increments the start offset timestamp by the advanceMsPerBatch for the end offset timestamp