SupportsPushDownFilters¶

SupportsPushDownFilters is an extension of the ScanBuilder abstraction for scan builders that can pushFilters and pushedFilters (for filter pushdown performance optimization and thus reduce the size of the data to be read).

Obsolete as of Spark 3.3.0

SupportsPushDownV2Filters is preferred as it uses the modern Predicate expression.

Contract¶

Filter[] pushedFilters()

Data source filters that were pushed down to the data source (in pushFilters)

Used when:

PushDownUtils is requested to pushFilters
V2ScanRelationPushDown logical optimization is executed (and getWrappedScan for V1Scans)

Filter[] pushFilters(
  Filter[] filters)

Data source filters that need to be evaluated again after scanning (so Spark can plan an extra filter operator)

Used when:

Note

No built-in implementations available.