DeltaSource

DeltaSource is…​FIXME

getBatch Method

getBatch(
  start: Option[Offset],
  end: Offset): DataFrame
getBatch is part of the Source contract (Spark Structured Streaming) to…​FIXME.

getBatch…​FIXME

getOffset Method

getOffset: Option[Offset]
getOffset is part of the streaming Source contract (Spark Structured Streaming) to get the maximum available offset for this source.

getOffset…​FIXME

getStartingOffset Internal Method

getStartingOffset(): Option[Offset]

getStartingOffset…​FIXME

getStartingOffset is used exclusively when DeltaSource is requested to getOffset.

getChangesWithRateLimit Internal Method

getChangesWithRateLimit(
  fromVersion: Long,
  fromIndex: Long,
  isStartingVersion: Boolean): Iterator[IndexedFile]

getChangesWithRateLimit…​FIXME

getChangesWithRateLimit is used when DeltaSource is requested to getStartingOffset and getOffset.

getChanges Internal Method

getChanges(
  fromVersion: Long,
  fromIndex: Long,
  isStartingVersion: Boolean): Iterator[IndexedFile]

getChanges…​FIXME

getChanges is used when DeltaSource is requested to getChangesWithRateLimit and getBatch.

Looking Up Snapshot By Version — getSnapshotAt Internal Method

getSnapshotAt(
  version: Long): Iterator[IndexedFile]

getSnapshotAt…​FIXME

getSnapshotAt is used exclusively when DeltaSource is requested to getChanges.