StateStoreId is a unique identifier of a StateStore with the following attributes:
- [[checkpointRootLocation]] Checkpoint Root Location - the root directory for state checkpointing
- [[operatorId]] Operator ID - a unique ID of the stateful operator
- [[partitionId]] Partition ID - the index of the partition
- [[storeName]] Store Name - the name of the state store (default: <
StateStoreId is <
StateStoreRDDis requested for the preferred locations of a partition (executed on the driver) and to compute it (later on an executor)
StateStoreProviderIdhelper object is requested to create a <
> (with a < > and the run ID of a streaming query) that is then used for the preferred locations of a partition of a
StateStoreAwareZipPartitionsRDD(executed on the driver) and to...FIXME
[[DEFAULT_STORE_NAME]] The name of the default state store (for reading state store data that was generated before store names were used, i.e. in Spark 2.2 and earlier) is default.
=== [[storeCheckpointLocation]] State Checkpoint Base Directory of Stateful Operator --
storeCheckpointLocation is Hadoop DFS's https://hadoop.apache.org/docs/r2.7.3/api/org/apache/hadoop/fs/Path.html[Path] of the checkpoint location (for the stateful operator by <
If the <
storeCheckpointLocation is used when
HDFSBackedStateStoreProvider is requested for the state checkpoint base directory.