PartitionGroup¶

Creating Instance¶

PartitionGroup takes the following to be created:

• LogContext
• Partition Queues (Map<TopicPartition, RecordQueue>)
• Lag Provider Function (Function<TopicPartition, OptionalLong>)
• recordLateness Sensor
• enforcedProcessing Sensor
• max.task.idle.ms configuration property

PartitionGroup is created when:

• StreamTask is created

Sensors¶

recordLateness¶

PartitionGroup is given the recordLateness metric sensor when created.

The recordLateness sensor is requested to record the following values:

• 0 when the event time advances (the timestamp of the current record is smaller than the current stream time)

• the difference (lateness) between the current stream time and the timestamp of the current otherwise

enforcedProcessing¶

PartitionGroup is given an enforcedProcessingSensor metric sensor when created.

Stream Time¶

PartitionGroup defines streamTime internal registry for the highest time (event-time watermark) across already-processed events (across all the partitions assigned to this StreamTask).

The streamTime is initially RecordQueue.UNKNOWN when PartitionGroup is created and later when cleared.

The streamTime changes to the given timestamp in setPartitionTime and next record.

The streamTime is used when:

• StreamTask is requested to maybePunctuateStreamTime and stream time

setPartitionTime¶

void setPartitionTime(
  TopicPartition partition, 
  long partitionTime)

setPartitionTime...FIXME

setPartitionTime is used when:

• StreamTask is requested to initializeTaskTime

Next (Stamped)Record¶

StampedRecord nextRecord(
  RecordInfo info, 
  long wallClockTime)

nextRecord...FIXME

nextRecord is used when:

• StreamTask is requested to process one record

Adding ConsumerRecords¶

int addRawRecords(
  TopicPartition partition,
  Iterable<ConsumerRecord<byte[], byte[]>> rawRecords)

addRawRecords...FIXME

addRawRecords is used when:

• StreamTask is requested to add records