Skip to content

AdmissionLimits

AdmissionLimits is used by DeltaSource to control how much data should be processed by a single micro-batch.

Creating Instance

AdmissionLimits takes the following to be created:

AdmissionLimits is created when:

Converting ReadLimit to AdmissionLimits

apply(
  limit: ReadLimit): Option[AdmissionLimits]

apply creates an AdmissionLimits for the given ReadLimit (Spark Structured Streaming).

ReadLimit AdmissionLimits
ReadAllAvailable None
ReadMaxFiles Maximum Number of Files
ReadMaxBytes Maximum Bytes
CompositeLimit Maximum Number of Files and Maximum Bytes

apply throws an UnsupportedOperationException for unknown ReadLimits:

Unknown ReadLimit: [limit]

apply is used when:

Admitting AddFile

admit(
  add: Option[AddFile]): Boolean

admit...FIXME

admit is used when:


Last update: 2021-03-24
Back to top