DiskBlockObjectWriter

DiskBlockObjectWriter is a custom java.io.OutputStream that BlockManager offers for writing data blocks to disk.

DiskBlockObjectWriter is used when:

Creating Instance

DiskBlockObjectWriter takes the following to be created:

DiskBlockObjectWriter is created when:

SerializationStream

DiskBlockObjectWriter manages a SerializationStream for writing a key-value record:

States

DiskBlockObjectWriter can be in the following states (that match the state of the underlying output streams):

  1. Initialized

  2. Open

  3. Closed

Writing Key and Value (of Record)

write(
  key: Any,
  value: Any): Unit

write opens the underlying stream unless open already.

write requests the SerializationStream to write the key and then the value.

In the end, write updates the write metrics.

write is used when:

commitAndGet Method

commitAndGet(): FileSegment

commitAndGet…​FIXME

commitAndGet is used when…​FIXME

Committing Writes and Closing Resources

close(): Unit

close…​FIXME

close is used when…​FIXME

revertPartialWritesAndClose Method

revertPartialWritesAndClose(): File

revertPartialWritesAndClose…​FIXME

revertPartialWritesAndClose is used when…​FIXME

updateBytesWritten Method

FIXME

initialize Method

FIXME

Writing Bytes (From Byte Array Starting From Offset)

write(kvBytes: Array[Byte], offs: Int, len: Int): Unit

write…​FIXME

FIXME

recordWritten Method

FIXME

Opening DiskBlockObjectWriter

open(): DiskBlockObjectWriter

open opens DiskBlockObjectWriter, i.e. initializes and re-sets bs and objOut internal output streams.

Internally, open makes sure that DiskBlockObjectWriter is not closed (i.e. hasBeenClosed flag is disabled). If it was, open throws a IllegalStateException:

Writer already closed. Cannot be reopened.

Unless DiskBlockObjectWriter has already been initialized (i.e. initialized flag is enabled), open initializes it (and turns initialized flag on).

Regardless of whether DiskBlockObjectWriter was already initialized or not, open requests SerializerManager to wrap mcs output stream for encryption and compression (for blockId) and sets it as bs.

open requests the SerializerInstance to serialize bs output stream and sets it as objOut.

open uses SerializerInstance that was specified when DiskBlockObjectWriter was created

In the end, open turns streamOpen flag on.

open is used exclusively when DiskBlockObjectWriter writes a key-value pair or bytes from a specified byte array but the stream is not open yet.

Internal Properties

Name Description

initialized

Internal flag…​FIXME

Used when…​FIXME

hasBeenClosed

Internal flag…​FIXME

Used when…​FIXME

mcs

FIXME

Used when…​FIXME

bs

FIXME

Used when…​FIXME