Skip to content

StateStoreId

[[creating-instance]] StateStoreId is a unique identifier of a StateStore with the following attributes:

  • [[checkpointRootLocation]] Checkpoint Root Location - the root directory for state checkpointing
  • [[operatorId]] Operator ID - a unique ID of the stateful operator
  • [[partitionId]] Partition ID - the index of the partition
  • [[storeName]] Store Name - the name of the state store (default: <>)

StateStoreId is <> when:

[[DEFAULT_STORE_NAME]] The name of the default state store (for reading state store data that was generated before store names were used, i.e. in Spark 2.2 and earlier) is default.

=== [[storeCheckpointLocation]] State Checkpoint Base Directory of Stateful Operator -- storeCheckpointLocation Method

[source, scala]

storeCheckpointLocation(): Path

storeCheckpointLocation is Hadoop DFS's https://hadoop.apache.org/docs/r2.7.3/api/org/apache/hadoop/fs/Path.html[Path] of the checkpoint location (for the stateful operator by <>, the partition by the <> in the <>).

If the <> is used (for Spark 2.2 and earlier), the <> is not included in the path.

storeCheckpointLocation is used when HDFSBackedStateStoreProvider is requested for the state checkpoint base directory.