Skip to content

RpcUtils

Maximum Message Size

maxMessageSizeBytes(
  conf: SparkConf): Int

maxMessageSizeBytes is the value of spark.rpc.message.maxSize configuration property in bytes (by multiplying the value by 1024 * 1024).

maxMessageSizeBytes throws an IllegalArgumentException when the value is above 2047 MB:

spark.rpc.message.maxSize should not be greater than 2047 MB

maxMessageSizeBytes is used when:

makeDriverRef

makeDriverRef(
  name: String,
  conf: SparkConf,
  rpcEnv: RpcEnv): RpcEndpointRef

makeDriverRef...FIXME

makeDriverRef is used when:


Last update: 2020-12-03