TriggeredGraphExecution¶

TriggeredGraphExecution is a GraphExecution that...FIXME

Creating Instance¶

TriggeredGraphExecution takes the following to be created:

• DataflowGraph
• PipelineUpdateContext
• onCompletion Callback (RunTerminationReason => Unit, default: _ => ())
• Clock (default: SystemClock)

TriggeredGraphExecution is created when:

• PipelineExecution is requested to startPipeline

Topological Execution Thread¶

topologicalExecutionThread: Option[Thread]

topologicalExecutionThread is the Topological Execution thread of execution of this pipeline.

topologicalExecutionThread is initialized and started when TriggeredGraphExecution is requested to start.

topologicalExecutionThread runs until awaitCompletion or stopInternal.

Start Pipeline¶

start(): Unit

start...FIXME

buildTopologicalExecutionThread¶

buildTopologicalExecutionThread(): Thread

buildTopologicalExecutionThread...FIXME

topologicalExecution¶

topologicalExecution(): Unit

topologicalExecution...FIXME

Streaming Trigger¶

streamTrigger(
  flow: Flow): Trigger

streamTrigger is AvailableNowTrigger (Spark Structured Streaming).

awaitCompletion¶

awaitCompletion(): Unit

awaitCompletion waits for this Topological Execution thread to die.

stopInternal¶

stopInternal(
  stopTopologicalExecutionThread: Boolean): Unit

stopInternal...FIXME

stopInternal is used when:

• TriggeredGraphExecution is requested to start and stop

Stop Execution¶

stop(): Unit

stop stopInternal (with stopTopologicalExecutionThread flag enabled).