Skip to content

InjectRuntimeFilter Logical Optimization

InjectRuntimeFilter is a logical optimization (i.e., a Rule of LogicalPlan).

InjectRuntimeFilter is part of InjectRuntimeFilter fixed-point batch of rules.

Executing Rule

apply(
  plan: LogicalPlan): LogicalPlan

apply is part of the Rule abstraction.


apply tryInjectRuntimeFilter unless one of the following holds (and the rule is a noop):

tryInjectRuntimeFilter

tryInjectRuntimeFilter(
  plan: LogicalPlan): LogicalPlan

tryInjectRuntimeFilter finds equi-joins in the given LogicalPlan.

When some requirements are met, tryInjectRuntimeFilter injectFilter on the left side first and on the right side if on the left was not successful.

tryInjectRuntimeFilter uses spark.sql.optimizer.runtimeFilter.number.threshold configuration property.

Injecting Filter Operator

injectFilter(
  filterApplicationSideExp: Expression,
  filterApplicationSidePlan: LogicalPlan,
  filterCreationSideExp: Expression,
  filterCreationSidePlan: LogicalPlan): LogicalPlan

injectFilter...FIXME

Injecting BloomFilter

injectBloomFilter(
  filterApplicationSideExp: Expression,
  filterApplicationSidePlan: LogicalPlan,
  filterCreationSideExp: Expression,
  filterCreationSidePlan: LogicalPlan): LogicalPlan

injectBloomFilter...FIXME

Note

injectBloomFilter is used when InjectRuntimeFilter is requested to inject a Filter with spark.sql.optimizer.runtime.bloomFilter.enabled configuration properties enabled.

injectInSubqueryFilter

injectInSubqueryFilter(
  filterApplicationSideExp: Expression,
  filterApplicationSidePlan: LogicalPlan,
  filterCreationSideExp: Expression,
  filterCreationSidePlan: LogicalPlan): LogicalPlan

injectInSubqueryFilter...FIXME

Note

injectInSubqueryFilter is used when InjectRuntimeFilter is requested to injectFilter with spark.sql.optimizer.runtime.bloomFilter.enabled configuration properties disabled (unlike spark.sql.optimizer.runtimeFilter.semiJoinReduction.enabled).