ReadLimit¶

ReadLimit is an abstraction of limits on how many records to read from a MicroBatchStream that SupportsAdmissionControl.

ReadLimit is used by stream execution engines when they are requested for unique streaming sources:

Factory Methods (Subset)¶

ReadLimit allAvailable()

allAvailable returns ReadAllAvailable.

allAvailable is used when:

SupportsAdmissionControl is requested for the default ReadLimit
KafkaMicroBatchStream is requested for the default ReadLimit
KafkaSource is requested for the default ReadLimit
AvailableNowDataStreamWrapper is requested to prepareForTriggerAvailableNow, getDefaultReadLimit
MicroBatchExecution is requested for the uniqueSources
ContinuousExecution is requested for the uniqueSources

ReadLimit compositeLimit(
  ReadLimit[] readLimits)

compositeLimit creates a CompositeReadLimit with the given ReadLimits.

Note

compositeLimit is used only in Kafka Data Source when minOffsetPerTrigger and maxOffsetsPerTrigger options are specified.

compositeLimit is used when:

ReadLimit maxRows(
  long rows)

maxRows creates a ReadMaxRows with the given rows.

maxRows is used when:

ReadLimit minRows(
  long rows,
  long maxTriggerDelayMs)

minRows creates a ReadMinRows with the given rows and maxTriggerDelayMs.

minRows is used when:

CompositeReadLimit is a ReadLimit that holds other ReadLimits.

CompositeReadLimit is created using ReadLimit.compositeLimit utility.

Used when:

ReadMaxRows is a ReadLimit that is described by the following:

Attribute	Description
`rows`	Approximate maximum rows to scan (maxRows)

ReadMaxRows is created using ReadLimit.maxRows utility.

Used when:

ReadMinRows is a ReadLimit that is described by the following:

Attribute	Description
`rows`	Approximate minimum rows to scan (minRows)
`maxTriggerDelayMs`	Approximate maximum trigger delay

ReadMinRows is created using ReadLimit.minRows utility.

Used when: