SparkConnectService¶
SparkConnectService is a BindableService (gRPC).
Creating Instance¶
SparkConnectService takes the following to be created:
-
debugflag
SparkConnectService is created when:
SparkConnectServiceis requested to startGRPCService
start¶
start(): Unit
start startGRPCService.
start is used when:
- SimpleSparkConnectService standalone application is started
SparkConnectPluginis requested to init- SparkConnectServer standalone application is started
startGRPCService¶
startGRPCService(): Unit
startGRPCService reads the values of the following configuration properties:
| Configuration Property | Default Value |
|---|---|
spark.connect.grpc.debug.enabled | true |
spark.connect.grpc.binding.port | 15002 |
startGRPCService builds a NettyServerBuilder with the spark.connect.grpc.binding.port and a SparkConnectService.
startGRPCService registers interceptors.
startGRPCService builds the server and starts it.
executePlan¶
gRPC Java
executePlan(
request: proto.ExecutePlanRequest,
responseObserver: StreamObserver[proto.ExecutePlanResponse]): Unit
executePlan is part of the SparkConnectServiceImplBase abstraction.
executePlan creates a SparkConnectStreamHandler (with the given StreamObserver) to let it handle the request.