BasicPythonArrowOutput¶
BasicPythonArrowOutput is a marker extension of the PythonArrowOutput abstraction for vectorized outputs of BasePythonRunners that produce ColumnarBatches (Spark SQL).
Implementations¶
- ArrowPythonRunner
CoGroupedArrowPythonRunner
Deserializing ColumnarBatch¶
PythonArrowOutput
deserializeColumnarBatch(
batch: ColumnarBatch,
schema: StructType): ColumnarBatch
deserializeColumnarBatch is part of the PythonArrowOutput abstraction.
deserializeColumnarBatch returns the given ColumnarBatch unchanged.