LowLatencyMemoryStream¶

LowLatencyMemoryStream is a MemoryStreamBaseClass that supports Real-Time Mode.

Creating Instance¶

LowLatencyMemoryStream takes the following to be created:

• ID
• SparkSession (Spark SQL)
• Number of partitions (default: 2)
• Clock (default: LowLatencyClock)

LowLatencyMemoryStream can be created using apply and singlePartition factory methods.

apply¶

apply[A: Encoder](
  implicit sparkSession: SparkSession): LowLatencyMemoryStream[A]
apply[A: Encoder](
  numPartitions: Int)(
  implicit sparkSession: SparkSession): LowLatencyMemoryStream[A]

apply...FIXME

singlePartition¶

singlePartition[A: Encoder](
  implicit sparkSession: SparkSession): LowLatencyMemoryStream[A]

singlePartition...FIXME

mergeOffsets¶

mergeOffsets(
  offsets: Array[PartitionOffset]): LowLatencyMemoryStreamOffset

mergeOffsets...FIXME

planInputPartitions¶

planInputPartitions(
  start: Offset): Array[InputPartition]

planInputPartitions...FIXME