PythonSQLMetrics¶

PythonSQLMetrics is a collection of SQL metrics of the physical operators in PySpark.

Performance Metrics¶

data returned from Python workers¶

data sent to Python workers¶

number of output rows¶

Implementations¶

• AggregateInPandasExec
• ArrowEvalPythonExec
• BatchEvalPythonExec
• FlatMapCoGroupsInPandasExec
• FlatMapGroupsInPandasExec
• MapInBatchExec
• StateStoreWriter
• WindowInPandasExec