BlockInfo

BlockInfo is a metadata of memory block — the memory block’s size, the number of readers and the id of the writer task.

BlockInfo has a StorageLevel, ClassTag and tellMaster flag.

Size

size attribute is the size of the memory block. It starts with 0.

It represents the number of bytes that BlockManager saved or BlockManager.doPutIterator.

Reader Count — readerCount Counter

readerCount counter is the number of readers of the memory block, i.e. the number of read locks. It starts with 0.

readerCount is incremented when a read lock is acquired and decreases when the following happens:

Writer Task — writerTask Attribute

writerTask attribute is the task that owns the write lock for the memory block.

A writer task can be one of the three possible identifiers:

  • NO_WRITER (i.e. -1) to denote no writers and hence no write lock in use.

  • NON_TASK_WRITER (i.e. -1024) for non-task threads, e.g. by a driver thread or by unit test code.

  • the task attempt id of the task which currently holds the write lock for this block.

The writer task is assigned in the following scenarios: