SupportsRuntimeFiltering¶

SupportsRuntimeFiltering is an extension of the SupportsRuntimeV2Filtering abstraction for connectors that want to filter initially planned InputPartitions using predicates Spark infers at runtime.

Contract¶

filterAttributes¶

NamedReference[] filterAttributes()

Used when:

• PartitionPruning logical optimization is executed (to getFilterableTableScan)
• RowLevelOperationRuntimeGroupFiltering logical optimization is executed

filter¶

void filter(
  Filter[] filters)
void filter(
  Predicate[] predicates) // (1)!

1. Converts the given Predicates to the older Filters

Used when:

• BatchScanExec physical operator is executed (to filteredPartitions)

Implementations¶