Skip to content

FlowAnalysis Utility

Singleton Object

FlowAnalysis is a Scala object which is a class that has exactly one instance. It is created lazily when it is referenced, like a lazy val.

Learn more in Tour of Scala.

createFlowFunctionFromLogicalPlan

createFlowFunctionFromLogicalPlan(
  plan: LogicalPlan): FlowFunction

createFlowFunctionFromLogicalPlan takes a LogicalPlan (that represents one of the supported logical commands) and creates a FlowFunction.

When executed, this FlowFunction creates a FlowAnalysisContext.

FlowFunction uses this FlowAnalysisContext to set the SQL configs (given to the FlowFunction being defined).

FlowFunction analyze this LogicalPlan (with the FlowAnalysisContext). This gives the result data (as a DataFrame).

In the end, FlowFunction creates a FlowFunctionResult with the result data (as a DataFrame) and the others (from the FlowAnalysisContext).

createFlowFunctionFromLogicalPlan is used when:

Analyze Logical Command

analyze(
  context: FlowAnalysisContext,
  plan: LogicalPlan): DataFrame

CTEs

analyze resolves pipeline-specific TVFs and CTEs.

SELECT ... FROM STREAM(t1)
SELECT ... FROM STREAM t1

Developers can define CTEs within their CREATE statements:

CREATE STREAMING TABLE a
WITH b AS (
   SELECT * FROM STREAM upstream
)
SELECT * FROM b

analyze...FIXME