JoinStateWatermarkPredicate¶

JoinStateWatermarkPredicate is an abstraction of join state watermark predicates.

A concrete JoinStateWatermarkPredicate is created using StreamingSymmetricHashJoinHelper utility (for planning a StreamingSymmetricHashJoinExec physical operator for execution with execution-specific configuration).

JoinStateWatermarkPredicate can be used to create a OneSideHashJoiner (and JoinStateWatermarkPredicates).

Sealed Trait

JoinStateWatermarkPredicate is a Scala sealed trait which means that all of the implementations are in the same compilation unit (a single file).

Learn more in the Scala Language Specification.

Contract¶

Description¶

desc: String

JoinStateWatermarkPredicate	Description
JoinStateKeyWatermarkPredicate	key predicate
JoinStateValueWatermarkPredicate	value predicate

Used when:

JoinStateWatermarkPredicate is requested for string representation

Watermark Expression¶

expr: Expression

A Catalyst Expression (Spark SQL)

Used when:

JoinStateWatermarkPredicate is requested for string representation
OneSideHashJoiner is created

Implementations¶

JoinStateKeyWatermarkPredicate¶

Watermark predicate on state keys (i.e., when the streaming watermark is defined either on the left or right join keys)

Created with a watermark expression (on left or right join keys) when:

StreamingSymmetricHashJoinHelper is requested for a JoinStateWatermarkPredicates for the left and right side of a stream-stream join (when IncrementalExecution is requested to optimize a query plan with a StreamingSymmetricHashJoinExec physical operator)

Used when:

OneSideHashJoiner is requested for the stateKeyWatermarkPredicateFunc (to remove an old state)

JoinStateValueWatermarkPredicate¶

Watermark predicate on state values

Created with a watermark expression (on left or right join keys) when:

StreamingSymmetricHashJoinHelper is requested for a JoinStateWatermarkPredicates for the left and right side of a stream-stream join (when IncrementalExecution is requested to optimize a query plan with a StreamingSymmetricHashJoinExec physical operator)

Used when:

OneSideHashJoiner is requested for the stateValueWatermarkPredicateFunc (to remove an old state)

String Representation¶

toString: String

toString is part of the java.lang.Object abstraction.

toString uses the desc and expr to build the following string representation:

[desc]: [expr]