Skip to content

Data Source Filter Predicate

Filter is the <> for <> that can be pushed down to a relation (aka data source).

Filter is used when:

[[contract]] [source, scala]

package org.apache.spark.sql.sources

abstract class Filter { // only required methods that have no implementation // the others follow def references: Array[String] }

.Filter Contract [cols="1,2",options="header",width="100%"] |=== | Method | Description

| references a| [[references]] Column references, i.e. list of column names that are referenced by a filter

Used when:

  • Filter is requested to <>

  • <>, <> and <> filters are requested for the <> |===

=== [[findReferences]] Finding Column References in Any Value -- findReferences Method

[source, scala]

findReferences(value: Any): Array[String]

findReferences takes the <> from the value filter is it is one or returns an empty array.