Skip to content

Serializer

Serializer is an abstraction of serializers for serialization and deserialization of tasks (closures) and data blocks in a Spark application.

Contract

Creating New SerializerInstance

newInstance(): SerializerInstance

Creates a new SerializerInstance

Used when:

  • Task is created (only used in tests)
  • SerializerSupport (Spark SQL) utility is used to newSerializer
  • RangePartitioner is requested to writeObject and readObject
  • TorrentBroadcast utility is used to blockifyObject and unBlockifyObject
  • TaskRunner is requested to run
  • NettyBlockRpcServer is requested to deserializeMetadata
  • NettyBlockTransferService is requested to uploadBlock
  • PairRDDFunctions is requested to...FIXME
  • ParallelCollectionPartition is requested to...FIXME
  • RDD is requested to...FIXME
  • ReliableCheckpointRDD utility is used to...FIXME
  • NettyRpcEnvFactory is requested to create a RpcEnv
  • DAGScheduler is created
  • others

Implementations

Accessing Serializer

Serializer is available using SparkEnv as the closureSerializer and serializer.

closureSerializer

SparkEnv.get.closureSerializer

serializer

SparkEnv.get.serializer

Serialized Objects Relocation Requirements

supportsRelocationOfSerializedObjects: Boolean

supportsRelocationOfSerializedObjects is disabled (false) by default.

supportsRelocationOfSerializedObjects is used when:


Last update: 2021-07-17
Back to top