RDDStorageInfo contains information about RDD persistence:

  • RDD id

  • RDD name

  • Number of RDD partitions

  • Number of cached RDD partitions

  • Storage level ID

  • Memory used

  • Disk used

  • Data distribution (as Seq[RDDDataDistribution])

  • Partitions (as Seq[RDDPartitionInfo]])

RDDStorageInfo is created exclusively when LiveRDD is requested to doUpdate (when requested to write).

RDDStorageInfo is used when:

  1. web UI’s StoragePage is requested to render an HTML table row and an entire table for RDD details

  2. REST API’s AbstractApplicationResource is requested for rddList (at storage/rdd path)

  3. AppStatusStore is requested for rddList