InjectRuntimeFilter Logical Optimization¶
InjectRuntimeFilter
is a logical optimization (i.e., a Rule of LogicalPlan).
InjectRuntimeFilter
is part of InjectRuntimeFilter fixed-point batch of rules.
Executing Rule¶
apply(
plan: LogicalPlan): LogicalPlan
apply
is part of the Rule abstraction.
apply
tryInjectRuntimeFilter unless one of the following holds (and the rule is a noop):
- The given query plan is a correlated
Subquery
- spark.sql.optimizer.runtimeFilter.semiJoinReduction.enabled and spark.sql.optimizer.runtime.bloomFilter.enabled are both disabled
tryInjectRuntimeFilter¶
tryInjectRuntimeFilter(
plan: LogicalPlan): LogicalPlan
tryInjectRuntimeFilter
finds equi-joins in the given LogicalPlan.
When some requirements are met, tryInjectRuntimeFilter
injectFilter on the left side first and on the right side if on the left was not successful.
tryInjectRuntimeFilter
uses spark.sql.optimizer.runtimeFilter.number.threshold configuration property.
Injecting Filter Operator¶
injectFilter(
filterApplicationSideExp: Expression,
filterApplicationSidePlan: LogicalPlan,
filterCreationSideExp: Expression,
filterCreationSidePlan: LogicalPlan): LogicalPlan
injectFilter
...FIXME
Injecting BloomFilter¶
injectBloomFilter(
filterApplicationSideExp: Expression,
filterApplicationSidePlan: LogicalPlan,
filterCreationSideExp: Expression,
filterCreationSidePlan: LogicalPlan): LogicalPlan
injectBloomFilter
...FIXME
Note
injectBloomFilter
is used when InjectRuntimeFilter
is requested to inject a Filter with spark.sql.optimizer.runtime.bloomFilter.enabled configuration properties enabled.
injectInSubqueryFilter¶
injectInSubqueryFilter(
filterApplicationSideExp: Expression,
filterApplicationSidePlan: LogicalPlan,
filterCreationSideExp: Expression,
filterCreationSidePlan: LogicalPlan): LogicalPlan
injectInSubqueryFilter
...FIXME
Note
injectInSubqueryFilter
is used when InjectRuntimeFilter
is requested to injectFilter with spark.sql.optimizer.runtime.bloomFilter.enabled configuration properties disabled (unlike spark.sql.optimizer.runtimeFilter.semiJoinReduction.enabled).