BatchTableWrite Flow Execution¶

BatchTableWrite is a FlowExecution that writes a batch DataFrame to a Table.

Creating Instance¶

BatchTableWrite takes the following to be created:

• TableIdentifier
• ResolvedFlow
• DataflowGraph
• Table
• PipelineUpdateContext
• Configuration Properties

BatchTableWrite is created when:

• FlowPlanner is requested to plan a CompleteFlow

Execute¶

executeInternal(): Future[Unit]

executeInternal activates the configuration properties in the current SparkSession.

executeInternal requests this PipelineUpdateContext for the FlowProgressEventLogger to recordRunning with this ResolvedFlow.

executeInternal requests this DataflowGraph to re-analyze this ResolvedFlow to get the DataFrame (the logical query plan)

executeInternal executes append batch write asynchronously:

1. Creates a DataFrameWriter for the batch query's logical plan (the DataFrame).
2. Sets the write format to the format of this Table.
3. In the end, executeInternal appends the rows to this Table (using DataFrameWriter.saveAsTable operator).

isStreaming¶

isStreaming: Boolean

isStreaming is always disabled (false).