SparkConnectPlugin¶
SparkConnectPlugin is a Spark driver plugin that starts a SparkConnectService upon initialization.
SparkConnectPlugin is a SparkPlugin (Spark Core) with the driver-side component only.
SparkConnectPlugin can be installed into Spark applications using spark.plugins (Spark Core) configuration property (e.g., Spark Connect Shell).
Driver-Side Component¶
SparkPlugin
driverPlugin is part of the SparkPlugin (Spark Core) abstraction.
driverPlugin creates a new DriverPlugin (Apache Spark) that does the following:
- Starts a SparkConnectService when requested to initialize
- Stops the SparkConnectService when requested to shut down