TriggeredGraphExecution¶
TriggeredGraphExecution
is a GraphExecution that...FIXME
Creating Instance¶
TriggeredGraphExecution
takes the following to be created:
- DataflowGraph
- PipelineUpdateContext
-
onCompletion
Callback (RunTerminationReason => Unit
, default:_ => ()
) -
Clock
(default:SystemClock
)
TriggeredGraphExecution
is created when:
PipelineExecution
is requested to startPipeline
Topological Execution Thread¶
topologicalExecutionThread: Option[Thread]
topologicalExecutionThread
is the Topological Execution thread of execution of this pipeline.
topologicalExecutionThread
is initialized and started when TriggeredGraphExecution
is requested to start.
topologicalExecutionThread
runs until awaitCompletion or stopInternal.
Start Pipeline¶
start
...FIXME
buildTopologicalExecutionThread¶
buildTopologicalExecutionThread(): Thread
buildTopologicalExecutionThread
...FIXME
topologicalExecution¶
topologicalExecution(): Unit
topologicalExecution
...FIXME
Streaming Trigger¶
GraphExecution
streamTrigger(
flow: Flow): Trigger
streamTrigger
is part of the GraphExecution abstraction.
streamTrigger
is AvailableNowTrigger
(Spark Structured Streaming).
awaitCompletion¶
awaitCompletion
waits for this Topological Execution thread to die.
stopInternal¶
stopInternal(
stopTopologicalExecutionThread: Boolean): Unit
stopInternal
...FIXME
stopInternal
is used when:
Stop Execution¶
stop
stopInternal (with stopTopologicalExecutionThread
flag enabled).