Skip to content

== [[EpochCoordinatorRef]] EpochCoordinatorRef

EpochCoordinatorRef is...FIXME

=== [[create]] Creating Remote Reference to EpochCoordinator RPC Endpoint -- create Factory Method

[source, scala]

create( writer: StreamWriter, reader: ContinuousReader, query: ContinuousExecution, epochCoordinatorId: String, startEpoch: Long, session: SparkSession, env: SparkEnv): RpcEndpointRef


create...FIXME

NOTE: create is used exclusively when ContinuousExecution is requested to <>.

=== [[get]] Getting Remote Reference to EpochCoordinator RPC Endpoint -- get Factory Method

[source, scala]

get(id: String, env: SparkEnv): RpcEndpointRef

get...FIXME

[NOTE]

get is used when:

  • DataSourceV2ScanExec leaf physical operator is requested for the input RDDs (and creates a <> for a ContinuousReader)

  • ContinuousQueuedDataReader is created (and initializes the <>)

  • EpochMarkerGenerator is created (and initializes the <>)

  • ContinuousWriteRDD is requested to <>

* WriteToContinuousDataSourceExec is requested to <>