Skip to content

GraphRegistrationContext

GraphRegistrationContext is a registry of tables, views, and flows in a pipeline (dataflow graph).

GraphRegistrationContext is required to create a new SqlGraphRegistrationContext.

Eventually, GraphRegistrationContext becomes a DataflowGraph (to create a PipelineUpdateContextImpl to run a pipeline).

Creating Instance

GraphRegistrationContext takes the following to be created:

  • Default Catalog
  • Default Database
  • Default SQL Configuration Properties

GraphRegistrationContext is created when:

Create DataflowGraph

toDataflowGraph: DataflowGraph

toDataflowGraph creates a new DataflowGraph with the tables, views, and flows fully-qualified, resolved, and de-duplicated.

AnalysisException

toDataflowGraph reports an AnalysisException for a GraphRegistrationContext with no tables and no PersistedViews (in the views registry).


toDataflowGraph is used when:

assertNoDuplicates

assertNoDuplicates(
  qualifiedTables: Seq[Table],
  validatedViews: Seq[View],
  qualifiedFlows: Seq[UnresolvedFlow]): Unit

assertNoDuplicates...FIXME

assertFlowIdentifierIsUnique

assertFlowIdentifierIsUnique(
  flow: UnresolvedFlow,
  datasetType: DatasetType,
  flows: Seq[UnresolvedFlow]): Unit

assertFlowIdentifierIsUnique throws an AnalysisException if the given UnresolvedFlow's identifier is used by multiple flows (among the given flows):

Flow [flow_name] was found in multiple datasets: [dataset_names]

Tables

GraphRegistrationContext creates an empty registry of Tables when created.

A new Table is added when registerTable.

Views

GraphRegistrationContext creates an empty registry of Views when created.

Flows

GraphRegistrationContext creates an empty registry of UnresolvedFlows when created.

Register Table

registerTable(
  tableDef: Table): Unit

registerTable adds the given Table to the tables registry.


registerTable is used when:

Register Flow

registerFlow(
  flowDef: UnresolvedFlow): Unit

registerFlow adds the given UnresolvedFlow to the flows registry.


registerFlow is used when: