InputPartition¶

InputPartition is an abstraction of input partitions in Connector API with optional location preferences.

InputPartition is a Java Serializable.

Contract¶

preferredLocations¶

String[] preferredLocations()

Specifies preferred locations (executor hosts)

By default, preferredLocations defines no location preferences (is simply empty).

See:

• FilePartition

Used when:

• FileScanRDD is requested for preferred locations
• DataSourceRDD is requested for preferred locations
• ContinuousDataSourceRDD (Spark Structured Streaming) is requested for preferred locations

Implementations¶

• ContinuousMemoryStreamInputPartition
• FilePartition
• HasPartitionKey
• KafkaBatchInputPartition
• KafkaContinuousInputPartition
• MemoryStreamInputPartition
• RatePerMicroBatchStreamInputPartition
• RateStreamContinuousInputPartition
• RateStreamMicroBatchInputPartition
• TextSocketContinuousInputPartition
• TextSocketInputPartition