Skip to content


StateStoreAwareZipPartitionsRDD is a ZippedPartitionsRDD2 with the <> and <> parent RDDs.

StateStoreAwareZipPartitionsRDD is <> when StreamingSymmetricHashJoinExec physical operator is requested to execute (and requests <> for one).

Creating Instance

StateStoreAwareZipPartitionsRDD takes the following to be created:

=== [[getPreferredLocations]] Placement Preferences of Partition (Preferred Locations) -- getPreferredLocations Method

[source, scala]

getPreferredLocations(partition: Partition): Seq[String]

NOTE: getPreferredLocations is a part of the RDD Contract to specify placement preferences (aka preferred task locations), i.e. where tasks should be executed to be as close to the data as possible.

getPreferredLocations simply requests the StateStoreCoordinatorRef for the location of every <> (with the <> and the partition ID) and returns unique executor IDs (so that processing a partition happens on the executor with the proper state store for the operator and the partition).