Skip to content

CachedDS — Cached Delta State

CachedDS is used when StateCache is requested to cacheDS.

Creating Instance

CachedDS takes the following to be created:

  • Dataset[A]
  • Name

CachedDS is created when:

  • StateCache is requested to cacheDS


cachedDs: Option[DatasetRefCache[Row]]


cachedDs is a value so initialized immediately when CachedDS is created.

cachedDs requests the Dataset for an RDD[InternalRow].

cachedDs associates the RDD with the name and marks it to be persist (on the first action).

cachedDs adds the RDD to the cached registry.


CachedDS is an internal class of StateCache and has access to its internals.

cachedDs is used when:

  • CachedDS is requested to getDF


getDS: Dataset[A]

getDS gets the cached DataFrame and converts the rows to A type.

getDS is used when: