Skip to content

MetadataLog

MetadataLog is an abstraction of metadata logs that can add, get, getLatest and purge metadata (of type T).

Type Constructor

MetadataLog[T] is a Scala type constructor with the type parameter T

Contract

Storing Metadata of Streaming Batch

add(
  batchId: Long,
  metadata: T): Boolean

Stores (adds) metadata of a streaming batch

Used when:

get

get(
  batchId: Long): Option[T]
get(
  startId: Option[Long],
  endId: Option[Long]): Array[(Long, T)]

Looks up (gets) metadata of one or more streaming batches

Used when...FIXME

getLatest

getLatest(): Option[(Long, T)]

Looks up the latest-committed metadata (if available)

Used when...FIXME

purge

purge(
  thresholdBatchId: Long): Unit

Purging (removing) metadata older than the given threshold

Used when...FIXME

Implementations