Skip to content

GroupStateImpl

GroupStateImpl is the default and only known GroupState in Spark Structured Streaming.

GroupStateImpl holds per-group <> of type S per group key.

GroupStateImpl is <> when GroupStateImpl helper object is requested for the following:

  • <>

  • <>

Creating Instance

GroupStateImpl takes the following to be created:

  • [[optionalValue]] State value (of type S)
  • [[batchProcessingTimeMs]] Batch processing time
  • [[eventTimeWatermarkMs]] eventTimeWatermarkMs
  • [[timeoutConf]] GroupStateTimeout
  • [[hasTimedOut]] hasTimedOut flag
  • [[watermarkPresent]] watermarkPresent flag

=== [[createForStreaming]] Creating GroupStateImpl for Streaming Query -- createForStreaming Factory Method

[source, scala]

createForStreamingS: GroupStateImpl[S]


createForStreaming simply creates a <> with the given input arguments.

NOTE: createForStreaming is used exclusively when InputProcessor is requested to callFunctionAndUpdateState (when InputProcessor is requested to processNewData and processTimedOutState).

=== [[createForBatch]] Creating GroupStateImpl for Batch Query -- createForBatch Factory Method

[source, scala]

createForBatch( timeoutConf: GroupStateTimeout, watermarkPresent: Boolean): GroupStateImpl[Any]


createForBatch...FIXME

NOTE: createForBatch is used when...FIXME

=== [[toString]] Textual Representation -- toString Method

[source, scala]

toString: String

NOTE: toString is part of the ++https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#toString--++[java.lang.Object] contract for the string representation of the object.

toString...FIXME

=== [[setTimeoutDuration]] Specifying Timeout Duration for ProcessingTimeTimeout -- setTimeoutDuration Method

[source, scala]

setTimeoutDuration(durationMs: Long): Unit

setTimeoutDuration...FIXME

setTimeoutDuration is part of the GroupState abstraction.

=== [[setTimeoutTimestamp]] Specifying Timeout Timestamp for EventTimeTimeout -- setTimeoutTimestamp Method

[source, scala]

setTimeoutTimestamp(durationMs: Long): Unit

setTimeoutTimestamp...FIXME

setTimeoutTimestamp is part of the GroupState abstraction.

=== [[getCurrentProcessingTimeMs]] Getting Processing Time -- getCurrentProcessingTimeMs Method

[source, scala]

getCurrentProcessingTimeMs(): Long

getCurrentProcessingTimeMs simply returns the <>.

getCurrentProcessingTimeMs is part of the GroupState abstraction.

=== [[update]] Updating State -- update Method

[source, scala]

update(newValue: S): Unit

update...FIXME

update is part of the GroupState abstraction.

=== [[remove]] Removing State -- remove Method

[source, scala]

remove(): Unit

remove...FIXME

remove is part of the GroupState abstraction.

=== [[internal-properties]] Internal Properties

[cols="30m,70",options="header",width="100%"] |=== | Name | Description

| value a| [[value]] FIXME

Used when...FIXME

| defined a| [[defined]] FIXME

Used when...FIXME

| updated a| [[updated]][[hasUpdated]] Updated flag that says whether the state has been <> or not

Default: false

Disabled (false) when GroupStateImpl is requested to <>

Enabled (true) when GroupStateImpl is requested to <>

| removed a| [[removed]][[hasRemoved]] Removed flag that says whether the state is marked <> or not

Default: false

Disabled (false) when GroupStateImpl is requested to <>

Enabled (true) when GroupStateImpl is requested to <>

| timeoutTimestamp a| [[timeoutTimestamp]][[getTimeoutTimestamp]] Current timeout timestamp (in millis) for GroupStateTimeout.EventTimeTimeout or GroupStateTimeout.ProcessingTimeTimeout

[[NO_TIMESTAMP]] Default: -1

Defined using <> (for EventTimeTimeout) and <> (for ProcessingTimeTimeout) |===