StreamingAggregationStateManagerBaseImpl¶
StreamingAggregationStateManagerBaseImpl
is the base implementation of the StreamingAggregationStateManager contract for state managers for streaming aggregations.
[[keyProjector]] StreamingAggregationStateManagerBaseImpl
uses UnsafeProjection
to <
[[implementations]] .StreamingAggregationStateManagerBaseImpls [cols="1,2",options="header",width="100%"] |=== | StreamingAggregationStateManagerBaseImpl | Description
| StreamingAggregationStateManagerImplV1 | [[StreamingAggregationStateManagerImplV1]] Legacy StreamingAggregationStateManager (used when spark.sql.streaming.aggregation.stateFormatVersion configuration property is 1
)
| StreamingAggregationStateManagerImplV2 | [[StreamingAggregationStateManagerImplV2]] Default StreamingAggregationStateManager (used when spark.sql.streaming.aggregation.stateFormatVersion configuration property is 2
) |===
[[creating-instance]] StreamingAggregationStateManagerBaseImpl
takes the following to be created:
- [[keyExpressions]] Catalyst expressions for the keys (
Seq[Attribute]
) - [[inputRowAttributes]] Catalyst expressions for the input rows (
Seq[Attribute]
)
NOTE: StreamingAggregationStateManagerBaseImpl
is a Scala abstract class and cannot be <
Committing (Changes to) State Store¶
commit(
store: StateStore): Long
commit
is part of the StreamingAggregationStateManager abstraction.
commit
simply requests the state store to commit state changes.
Removing Key From State Store¶
remove(
store: StateStore,
key: UnsafeRow): Unit
remove
is part of the StreamingAggregationStateManager abstraction.
remove
...FIXME
getKey¶
getKey(
row: UnsafeRow): UnsafeRow
getKey
is part of the StreamingAggregationStateManager abstraction.
getKey
...FIXME
Getting All Keys in State Store¶
keys(
store: StateStore): Iterator[UnsafeRow]
keys
is part of the StreamingAggregationStateManager abstraction.
keys
...FIXME