Skip to content

DataWriter¶

DataWriter is an abstraction of data writers (of data of type T).

Contract¶

Aborting Write¶

void abort()

See:

KafkaDataWriter

Used when:

FileFormatWriter utility is used to executeTask
DataWritingSparkTask utility is used to process a partition
ContinuousWriteRDD (Spark Structured Streaming) is requested to compute a partition

Committing Successful Write¶

WriterCommitMessage commit()

See:

KafkaDataWriter

Used when:

DataWritingSparkTask utility is used to process a partition
ContinuousWriteRDD (Spark Structured Streaming) is requested to compute a partition

currentMetricsValues¶

CustomTaskMetric[] currentMetricsValues()

See:

KafkaDataWriter

Used when:

FileFormatWriter utility is used to executeTask
DataWritingSparkTask utility is used to process a partition
ContinuousWriteRDD (Spark Structured Streaming) is requested to compute a partition

Writing Out Record¶

void write(
  T record)

See:

KafkaDataWriter

Used when:

DataWritingSparkTask is requested to write a record out
ContinuousWriteRDD (Spark Structured Streaming) is requested to compute a partition

Implementations¶