EngineContext¶

EngineContext is the execution context of ksqlDB engine.

Creating Instance¶

EngineContext takes the following to be created:

• ServiceContext
• ProcessingLogContext
• MutableMetaStore
• QueryIdGenerator
• KsqlParser
• QueryCleanupService
• KsqlConfig
• QueryRegistry
• RuntimeAssignor

EngineContext is created using create and createSandbox factories.

ProcessingLogContext¶

EngineContext is given a ProcessingLogContext when created.

The ProcessingLogContext is used when:

• getProcessingLogContext
• Creating a QueryEngine

DdlCommandExec¶

EngineContext creates a DdlCommandExec when created.

The DdlCommandExec is used to execute DDL commands.

QueryRegistry¶

EngineContext is given a QueryRegistry when created.

The QueryRegistry is used when:

• createSandbox
• maybeTerminateCreateAsQuery
• throwIfInsertQueriesExist

getQueryRegistry¶

QueryRegistry getQueryRegistry()

getQueryRegistry is used when:

• EngineExecutor is requested to executeTransientQuery, executeStreamPullQuery, sourceTablePlan, plan a statement, planQuery, executePersistentQuery
• KsqlEngine is requested to...FIXME
• PullQueryExecutionUtil is requested to findMaterializingQuery
• QueryIdUtil is requested to buildId
• SandboxedExecutionContext is requested to...FIXME
• ScalablePushQueryExecutionUtil is requested to...FIXME

Creating QueryEngine¶

QueryEngine createQueryEngine(
  ServiceContext serviceContext)

createQueryEngine creates a QueryEngine (with the given ServiceContext and the ProcessingLogContext).

createQueryEngine is simply a convenient factory method that does nothing but new QueryEngine and that little programming trick makes for a more readable fluent client code.

EngineContext
  .createQueryEngine(...)
  .buildPhysicalPlan(...)

createQueryEngine is used when:

• EngineExecutor is requested to plan a Query for execution

CommandFactories¶

EngineContext creates a CommandFactories when created.

The CommandFactories is used to create a DdlCommand.

Parsing SQL Statements¶

List<ParsedStatement> parse(
  String sql)

parse requests the KsqlParser to parse the given SQL statements.

parse is used when:

• EngineContext is requested to substituteVariables
• KsqlEngine is requested to parse a SQL text
• SandboxedExecutionContext is requested to parse

Preparing Statement for Execution¶

PreparedStatement<?> prepare(
  ParsedStatement stmt,
  Map<String, String> variablesMap)

prepare substitutes variables (in the given ParsedStatement with the variablesMap) and then requests the KsqlParser to prepare the ParsedStatement (with the variables resolved).

prepare sanitizes the statement based on the following configuration properties (in the KsqlConfig):

• ksql.lambdas.enabled
• ksql.rowpartition.rowoffset.enabled

In the end, prepare creates a new PreparedStatement (with an unmasked KSQL statement in text format and the sanitized Statement).

prepare is used when (the available KsqlExecutionContexts are requested to prepare a KSQL statement for execution):

• KsqlEngine
• SandboxedExecutionContext

Variable Substitution¶

ParsedStatement substituteVariables(
  ParsedStatement stmt,
  Map<String, String> variablesMap)

substituteVariables substitutes variables (in the given ParsedStatement with the variablesMap) and then parses the SQL text.

Creating EngineContext¶

EngineContext create(
  ServiceContext serviceContext,
  ProcessingLogContext processingLogContext,
  MutableMetaStore metaStore,
  QueryIdGenerator queryIdGenerator,
  QueryCleanupService cleanupService,
  KsqlConfig ksqlConfig,
  Collection<QueryEventListener> registrationListeners)

create creates an EngineContext (with a new DefaultKsqlParser, a new QueryRegistryImpl and the others).

create is used when:

• KsqlEngine is created

Creating Sandboxed EngineContext¶

EngineContext createSandbox(
  ServiceContext serviceContext)

createSandbox creates an EngineContext (with a SandboxedServiceContext, a new DefaultKsqlParser, and the others sandboxed).

createSandbox is used when:

• SandboxedExecutionContext is created

Creating DdlCommand¶

DdlCommand createDdlCommand(
  KsqlStructuredDataOutputNode outputNode)
DdlCommand createDdlCommand(
  String sqlExpression,
  ExecutableDdlStatement statement,
  SessionConfig config)

createDdlCommand requests the CommandFactories to create a DdlCommand.

createDdlCommand is used when:

• EngineExecutor is requested to plan an ExecutableDdlStatement and maybeCreateSinkDdl

Executing DDL Command¶

String executeDdl(
  String sqlExpression,
  DdlCommand command,
  boolean withQuery,
  Set<SourceName> withQuerySources,
  boolean restoreInProgress)

executeDdl requests the DdlCommandExec to execute the DDL command.

executeDdl is used when:

• EngineExecutor is requested to execute a DDL Command