Skip to content

StreamingAggregationStateManagerBaseImpl

StreamingAggregationStateManagerBaseImpl is the base implementation of the StreamingAggregationStateManager contract for state managers for streaming aggregations.

[[keyProjector]] StreamingAggregationStateManagerBaseImpl uses UnsafeProjection to <>.

[[implementations]] .StreamingAggregationStateManagerBaseImpls [cols="1,2",options="header",width="100%"] |=== | StreamingAggregationStateManagerBaseImpl | Description

| StreamingAggregationStateManagerImplV1 | [[StreamingAggregationStateManagerImplV1]] Legacy StreamingAggregationStateManager (used when spark.sql.streaming.aggregation.stateFormatVersion configuration property is 1)

| StreamingAggregationStateManagerImplV2 | [[StreamingAggregationStateManagerImplV2]] Default StreamingAggregationStateManager (used when spark.sql.streaming.aggregation.stateFormatVersion configuration property is 2) |===

[[creating-instance]] StreamingAggregationStateManagerBaseImpl takes the following to be created:

  • [[keyExpressions]] Catalyst expressions for the keys (Seq[Attribute])
  • [[inputRowAttributes]] Catalyst expressions for the input rows (Seq[Attribute])

NOTE: StreamingAggregationStateManagerBaseImpl is a Scala abstract class and cannot be <> directly. It is created indirectly for the <>.

Committing (Changes to) State Store

commit(
  store: StateStore): Long

commit is part of the StreamingAggregationStateManager abstraction.

commit simply requests the state store to commit state changes.

Removing Key From State Store

remove(
  store: StateStore,
  key: UnsafeRow): Unit

remove is part of the StreamingAggregationStateManager abstraction.

remove...FIXME

getKey

getKey(
  row: UnsafeRow): UnsafeRow

getKey is part of the StreamingAggregationStateManager abstraction.

getKey...FIXME

Getting All Keys in State Store

keys(
  store: StateStore): Iterator[UnsafeRow]

keys is part of the StreamingAggregationStateManager abstraction.

keys...FIXME

Back to top