SupportsPushDownFilters¶
SupportsPushDownFilters
is an extension of the ScanBuilder abstraction for scan builders that can pushFilters and pushedFilters (for filter pushdown performance optimization and thus reduce the size of the data to be read).
Contract¶
pushedFilters¶
Filter[] pushedFilters()
Data source filters that were pushed down to the data source (in pushFilters)
Used when V2ScanRelationPushDown logical optimization is executed (that uses PushDownUtils
utility to pushFilters
)
pushFilters¶
Filter[] pushFilters(
Filter[] filters)
Data source filters that need to be evaluated again after scanning (so Spark can plan an extra filter operator)
Used when:
- V2ScanRelationPushDown logical optimization is executed (that uses
PushDownUtils
utility topushFilters
) CSVScanBuilder
is requested for a ScanOrcScanBuilder
is requested for a Scan