GroupStateImpl¶

GroupStateImpl is the default and only known GroupState in Spark Structured Streaming.

GroupStateImpl holds per-group <> of type S per group key.

GroupStateImpl is <> when GroupStateImpl helper object is requested for the following:

<>
<>

Creating Instance¶

GroupStateImpl takes the following to be created:

[[optionalValue]] State value (of type S)
[[batchProcessingTimeMs]] Batch processing time
[[eventTimeWatermarkMs]] eventTimeWatermarkMs
[[timeoutConf]] GroupStateTimeout
[[hasTimedOut]] hasTimedOut flag
[[watermarkPresent]] watermarkPresent flag

=== [[createForStreaming]] Creating GroupStateImpl for Streaming Query -- createForStreaming Factory Method

[source, scala]¶

createForStreamingS: GroupStateImpl[S]

createForStreaming simply creates a <> with the given input arguments.

NOTE: createForStreaming is used exclusively when InputProcessor is requested to callFunctionAndUpdateState (when InputProcessor is requested to processNewData and processTimedOutState).

=== [[createForBatch]] Creating GroupStateImpl for Batch Query -- createForBatch Factory Method

[source, scala]¶

createForBatch( timeoutConf: GroupStateTimeout, watermarkPresent: Boolean): GroupStateImpl[Any]

createForBatch...FIXME

NOTE: createForBatch is used when...FIXME

=== [[toString]] Textual Representation -- toString Method

[source, scala]¶

toString: String¶

NOTE: toString is part of the ++https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#toString--++[java.lang.Object] contract for the string representation of the object.

toString...FIXME

=== [[setTimeoutDuration]] Specifying Timeout Duration for ProcessingTimeTimeout -- setTimeoutDuration Method

[source, scala]¶

setTimeoutDuration(durationMs: Long): Unit¶

setTimeoutDuration...FIXME

setTimeoutDuration is part of the GroupState abstraction.

=== [[setTimeoutTimestamp]] Specifying Timeout Timestamp for EventTimeTimeout -- setTimeoutTimestamp Method

[source, scala]¶

setTimeoutTimestamp(durationMs: Long): Unit¶

setTimeoutTimestamp...FIXME

setTimeoutTimestamp is part of the GroupState abstraction.

=== [[getCurrentProcessingTimeMs]] Getting Processing Time -- getCurrentProcessingTimeMs Method

[source, scala]¶

getCurrentProcessingTimeMs(): Long¶

getCurrentProcessingTimeMs simply returns the <>.

getCurrentProcessingTimeMs is part of the GroupState abstraction.

=== [[update]] Updating State -- update Method

[source, scala]¶

update(newValue: S): Unit¶

update...FIXME

update is part of the GroupState abstraction.

=== [[remove]] Removing State -- remove Method

[source, scala]¶

remove(): Unit¶

remove...FIXME

remove is part of the GroupState abstraction.

=== [[internal-properties]] Internal Properties

[cols="30m,70",options="header",width="100%"] |=== | Name | Description

| value a| [[value]] FIXME

Used when...FIXME

| defined a| [[defined]] FIXME

Used when...FIXME

| updated a| [[updated]][[hasUpdated]] Updated flag that says whether the state has been <> or not

Default: false

Disabled (false) when GroupStateImpl is requested to <>

Enabled (true) when GroupStateImpl is requested to <>

| removed a| [[removed]][[hasRemoved]] Removed flag that says whether the state is marked <> or not

Default: false

Disabled (false) when GroupStateImpl is requested to <>

Enabled (true) when GroupStateImpl is requested to <>

| timeoutTimestamp a| [[timeoutTimestamp]][[getTimeoutTimestamp]] Current timeout timestamp (in millis) for GroupStateTimeout.EventTimeTimeout or GroupStateTimeout.ProcessingTimeTimeout

[[NO_TIMESTAMP]] Default: -1

Defined using <> (for EventTimeTimeout) and <> (for ProcessingTimeTimeout) |===