RatePerMicroBatchStream is a MicroBatchStream that SupportsTriggerAvailableNow.
RatePerMicroBatchStream takes the following to be created:
RatePerMicroBatchStream is created when:
isTriggerAvailableNow flag to determine whether it is executed in Trigger.AvailableNow mode or not (based on prepareForTriggerAvailableNow).
isTriggerAvailableNow flag is off (
isTriggerAvailableNow is used when:
prepareForTriggerAvailableNow is part of the SupportsTriggerAvailableNow abstraction.
prepareForTriggerAvailableNow turns the isTriggerAvailableNow flag on (for Trigger.AvailableNow mode).
limit: ReadLimit): Offset
latestOffset is part of the SupportsAdmissionControl abstraction.
latestOffset is different whether it is executed in Trigger.AvailableNow mode or not (based on isTriggerAvailableNow flag).
latestOffset calculates next offset just once for Trigger.AvailableNow and the offsetForTriggerAvailableNow registry has no value assigned yet.
In other words, for isTriggerAvailableNow,
latestOffset returns the value of the offsetForTriggerAvailableNow registry (that remains the same for every
For all the other triggers (when isTriggerAvailableNow is disabled),
latestOffset calculates next offset.
Calculating Next Offset
start: Offset): Offset
calculateNextOffset extractOffsetAndTimestamp from the given start Offset.
calculateNextOffset creates a
RatePerMicroBatchStreamOffset for the end offset (that is rowsPerBatch rows long):
- Increments the start offset by the rowsPerBatch for the end offset
- Increments the start offset timestamp by the advanceMsPerBatch for the end offset timestamp