Skip to content

= RetryingBlockFetcher

RetryingBlockFetcher is...FIXME

RetryingBlockFetcher is <> and immediately <> when:

  • NettyBlockTransferService is requested to storage:NettyBlockTransferService.md#fetchBlocks[fetchBlocks] (when network:TransportConf.md#io.maxRetries[maxIORetries] is greater than 0 which it is by default)

  • ExternalShuffleClient is requested to storage:ExternalShuffleClient.md#fetchBlocks[fetchBlocks] (when network:TransportConf.md#io.maxRetries[maxIORetries] is greater than 0 which it is by default)

RetryingBlockFetcher uses a <> to core:BlockFetchStarter.md#createAndStart[createAndStart] when requested to <> and later <>.

[[outstandingBlocksIds]] RetryingBlockFetcher uses outstandingBlocksIds internal registry of outstanding block IDs to fetch that is initially the <> when <>.

At <>, RetryingBlockFetcher prints out the following INFO message to the logs (with the number of <>):

Retrying fetch ([retryCount]/[maxRetries]) for [size] outstanding blocks after [retryWaitTime] ms

On <> and <>, <> removes the block ID from <>.

[[currentListener]] RetryingBlockFetcher uses a <> to remove block IDs from the <> internal registry.

== [[creating-instance]] Creating RetryingBlockFetcher Instance

RetryingBlockFetcher takes the following when created:

  • [[conf]] network:TransportConf.md[]
  • [[fetchStarter]] core:BlockFetchStarter.md[]
  • [[blockIds]] Block IDs to fetch
  • [[listener]] core:BlockFetchingListener.md[]

== [[start]] Starting RetryingBlockFetcher -- start Method

[source, java]

void start()

start simply <>.

[NOTE]

start is used when:

  • NettyBlockTransferService is requested to storage:NettyBlockTransferService.md#fetchBlocks[fetchBlocks] (when network:TransportConf.md#io.maxRetries[maxIORetries] is greater than 0 which it is by default)

* ExternalShuffleClient is requested to storage:ExternalShuffleClient.md#fetchBlocks[fetchBlocks] (when network:TransportConf.md#io.maxRetries[maxIORetries] is greater than 0 which it is by default)

== [[initiateRetry]] initiateRetry Internal Method

[source, java]

synchronized void initiateRetry()

initiateRetry...FIXME

[NOTE]

initiateRetry is used when:

  • RetryingBlockFetcher is requested to <>

* RetryingBlockFetchListener is requested to <>

== [[fetchAllOutstanding]] fetchAllOutstanding Internal Method

[source, java]

void fetchAllOutstanding()

fetchAllOutstanding requests <> to core:BlockFetchStarter.md#createAndStart[createAndStart] for the <>.

NOTE: fetchAllOutstanding is used when RetryingBlockFetcher is requested to <> and <>.

== [[RetryingBlockFetchListener]] RetryingBlockFetchListener

RetryingBlockFetchListener is a core:BlockFetchingListener.md[] that <> uses to remove block IDs from the <> internal registry.

=== [[RetryingBlockFetchListener-onBlockFetchSuccess]] onBlockFetchSuccess Method

[source, scala]

void onBlockFetchSuccess(String blockId, ManagedBuffer data)

NOTE: onBlockFetchSuccess is part of core:BlockFetchingListener.md#onBlockFetchSuccess[BlockFetchingListener Contract].

onBlockFetchSuccess...FIXME

=== [[RetryingBlockFetchListener-onBlockFetchFailure]] onBlockFetchFailure Method

[source, scala]

void onBlockFetchFailure(String blockId, Throwable exception)

NOTE: onBlockFetchFailure is part of core:BlockFetchingListener.md#onBlockFetchFailure[BlockFetchingListener Contract].

onBlockFetchFailure...FIXME


Last update: 2020-10-06