Skip to content

StreamingTableWrite

StreamingTableWrite is a StreamingFlowExecution.

When executed, StreamingTableWrite starts a streaming query to append new rows to an output table.

Creating Instance

StreamingTableWrite takes the following to be created:

StreamingTableWrite is created when:

Execute Streaming Query

StreamingFlowExecution
startStream(): StreamingQuery

startStream is part of the StreamingFlowExecution abstraction.

startStream builds the logical query plan of this flow's structured query (requesting the DataflowGraph to reanalyze this flow).

startStream creates a DataStreamWriter (Spark Structured Streaming) with the following:

DataStreamWriter's Property Value
queryName This displayName
checkpointLocation option This checkpoint path
trigger This streaming trigger
outputMode Append (always)
format The format of this output table (only when defined)

In the end, startStream starts the streaming write query to this output table.