Skip to content


= NettyBlockRpcServer

NettyBlockRpcServer is an[] to handle <> for[NettyBlockTransferService].

.NettyBlockRpcServer and NettyBlockTransferService image::NettyBlockRpcServer.png[align="center"]

== [[creating-instance]] Creating Instance

NettyBlockRpcServer takes the following to be created:

  • [[appId]] Application ID
  • [[serializer]][]
  • [[blockManager]][]

NettyBlockRpcServer is created when NettyBlockTransferService is requested to[initialize].

== [[streamManager]] OneForOneStreamManager

NettyBlockRpcServer uses a[] for...FIXME

== [[receive]] Receiving RPC Messages

[source, scala]

receive( client: TransportClient, rpcMessage: ByteBuffer, responseContext: RpcResponseCallback): Unit


receive is part of[RpcHandler] abstraction.

== [[messages]] Messages

=== [[OpenBlocks]] OpenBlocks


OpenBlocks( String appId, String execId, String[] blockIds)

When received, NettyBlockRpcServer requests the <> for[block data] for every block id in the message. The block data is a collection of[] for every block id in the incoming message.

NettyBlockRpcServer then[registers a stream of ManagedBuffers (for the blocks) with the internal StreamManager] under streamId.

NOTE: The internal StreamManager is[OneForOneStreamManager] and is created when <>.

You should see the following TRACE message in the logs:


NettyBlockRpcServer: Registered streamId [streamId] with [size] buffers

In the end, NettyBlockRpcServer responds with a StreamHandle (with the streamId and the number of blocks). The response is serialized as a ByteBuffer.

Posted when OneForOneBlockFetcher is requested to[start].

=== [[UploadBlock]] UploadBlock


UploadBlock( String appId, String execId, String blockId, byte[] metadata, byte[] blockData)

When received, NettyBlockRpcServer deserializes the metadata of the input message to get the[StorageLevel] and ClassTag of the block being uploaded.

NettyBlockRpcServer creates a[] for the block id and requests the <> to[store the block].

In the end, NettyBlockRpcServer responds with a 0-capacity ByteBuffer.

Posted when NettyBlockTransferService is requested to[upload a block].

== [[receiveStream]] Receiving RPC Message with Streamed Data

[source, scala]

receiveStream( client: TransportClient, messageHeader: ByteBuffer, responseContext: RpcResponseCallback): StreamCallbackWithID


receiveStream is part of[RpcHandler] abstraction.

== [[logging]] Logging

Enable ALL logging level for logger to see what happens inside.

Add the following line to conf/


Refer to[Logging].

Last update: 2020-11-27