Skip to content

CachedDS — Cached Delta State

CachedDS is used when StateCache is requested to cacheDS.

Creating Instance

CachedDS takes the following to be created:

  • Dataset[A]
  • Name

CachedDS is created when:

  • StateCache is requested to cacheDS

cachedDs

cachedDs: Option[DatasetRefCache[Row]]

Note

cachedDs is a value so initialized immediately when CachedDS is created.

cachedDs requests the Dataset for an RDD[InternalRow].

cachedDs associates the RDD with the name and marks it to be persist (on the first action).

cachedDs adds the RDD to the cached registry.

Note

CachedDS is an internal class of StateCache and has access to its internals.

cachedDs is used when:

  • CachedDS is requested to getDF

getDS

getDS: Dataset[A]

getDS gets the cached DataFrame and converts the rows to A type.

getDS is used when: