OutputWriter¶

OutputWriter is an abstraction of output writers that write rows to a file system.

Contract¶

Closing¶

close(): Unit

Closes this OutputWriter

Used when:

• FileFormatDataWriter is requested to releaseCurrentWriter
• DynamicPartitionDataConcurrentWriter is requested to releaseResources

Path¶

path(): String

The file path to write records to

Used when:

• FileFormatDataWriter is requested to releaseCurrentWriter
• SingleDirectoryDataWriter is requested to write
• BaseDynamicPartitionDataWriter is requested to writeRecord

Writing Row Out¶

write(
  row: InternalRow): Unit

Writes out a single InternalRow

Used when:

• SingleDirectoryDataWriter is requested to write
• BaseDynamicPartitionDataWriter is requested to writeRecord

Implementations¶

• AvroOutputWriter
• CsvOutputWriter
• HiveOutputWriter
• JsonOutputWriter
• LibSVMOutputWriter
• OrcOutputWriter
• OrcOutputWriter
• ParquetOutputWriter
• TextOutputWriter