Skip to content

Columnar Processing

New in 3.0.0

Columnar Processing (Columnar Execution) introduces the following main abstractions:

Whole-Stage Java Code Generation

Columnar Processing is similar and a kind of "opposite" at the same time to Whole-Stage Java Code Generation (which is row-based). It is assumed that if a plan supports columnar execution, it can't support whole-stage-codegen at the same time (see the comment in the source code).

Columnar Processing was introduced to Apache Spark 3.0.0 as SPARK-27396.