Skip to content

Plugin Framework

Plugin Framework is an API for registering custom extensions (plugins) to be executed on the driver and executors.

Plugin Framework uses separate PluginContainers for the driver and executors, and spark.plugins configuration property for SparkPlugins to be registered.

Plugin Framework was introduced in Spark 2.4.4 (with an API for executors) with further changes in Spark 3.0.0 (to cover the driver).

Resources

Back to top