WriteTaskStatsTracker¶

WriteTaskStatsTracker is an abstraction of WriteTaskStatsTrackers that are notified about and can collect the WriteTaskStats about files, partitions and rows processed.

Contract¶

Closing File¶

closeFile(
  filePath: String): Unit

Used when:

• FileFormatDataWriter is requested to releaseCurrentWriter

Final WriteTaskStats¶

getFinalStats(
  taskCommitTime: Long): WriteTaskStats

Creates a WriteTaskStats

Used when:

• FileFormatDataWriter is requested to commit a successful write

New File¶

newFile(
  filePath: String): Unit

Used when:

• SingleDirectoryDataWriter is requested to newOutputWriter
• BaseDynamicPartitionDataWriter is requested to renewCurrentWriter

New Partition¶

newPartition(
  partitionValues: InternalRow): Unit

Used when:

• DynamicPartitionDataSingleWriter is requested to write out a record
• DynamicPartitionDataConcurrentWriter is requested to write out a record

New Row¶

newRow(
  filePath: String,
  row: InternalRow): Unit

Used when:

• SingleDirectoryDataWriter is requested to write out a record
• BaseDynamicPartitionDataWriter is requested to write a record out

Implementations¶

• BasicWriteTaskStatsTracker