Bloom Filter Join¶

Bloom Filter Join is an optimization of join queries by pre-filtering one side of a join using BloomFilter based on the values from the other side of the join.

Bloom Filter Join uses BloomFilters as runtime filters when spark.sql.optimizer.runtime.bloomFilter.enabled configuration property is enabled.

Bloom Filter Join uses InjectRuntimeFilter logical optimization to inject up to spark.sql.optimizer.runtimeFilter.number.threshold filters (BloomFilters or InSubquerys).

Configuration Properties¶

• spark.sql.optimizer.runtime.bloomFilter.enabled
• spark.sql.optimizer.runtime.bloomFilter.creationSideThreshold
• spark.sql.optimizer.runtimeFilter.number.threshold