DiskBlockObjectWriter is used when:
DiskBlockObjectWriter takes the following to be created:
DiskBlockObjectWriter is created when:
DiskBlockObjectWriter can be in the following states (that match the state of the underlying output streams):
write( key: Any, value: Any): Unit
In the end, write updates the write metrics.
write is used when:
BypassMergeSortShuffleWriter is requested to write records of a partition
ExternalAppendOnlyMap is requested to spillMemoryIteratorToDisk
ExternalSorter is requested to write all records into a partitioned file
SpillableIterator is requested to spill
WritablePartitionedPairCollection is requested for a destructiveSortedWritablePartitionedIterator
commitAndGet is used when…FIXME
revertPartialWritesAndClose is used when…FIXME
write(kvBytes: Array[Byte], offs: Int, len: Int): Unit
open makes sure that DiskBlockObjectWriter is not closed (i.e. hasBeenClosed flag is disabled). If it was,
open throws a
Writer already closed. Cannot be reopened.
Regardless of whether DiskBlockObjectWriter was already initialized or not,
SerializerManager to wrap
mcs output stream for encryption and compression (for blockId) and sets it as bs.
In the end,
open turns streamOpen flag on.