Skip to content

LogStore (io.delta.storage)

LogStore is an abstraction of transaction log stores (to read and write Delta log files).

LogStore is created for LogStoreAdaptor.

io.delta.storage

LogStore is part of io.delta.storage package meant for Delta Lake developers.

Note

There is another internal LogStore in org.apache.spark.sql.delta.storage package.

Contract

isPartialWriteVisible

Boolean isPartialWriteVisible(
  Path path,
  Configuration hadoopConf)

Used when:

listFrom

Iterator<FileStatus> listFrom(
  Path path,
  Configuration hadoopConf) throws FileNotFoundException

Used when:

  • LogStoreAdaptor is requested to listFrom

read

CloseableIterator<String> read(
  Path path,
  Configuration hadoopConf)

Used when:

resolvePathOnPhysicalStorage

Path resolvePathOnPhysicalStorage(
  Path path,
  Configuration hadoopConf)

Used when:

write

void write(
  Path path,
  Iterator<String> actions,
  Boolean overwrite,
  Configuration hadoopConf) throws FileAlreadyExistsException

Used when:

  • LogStoreAdaptor is requested to write

Creating Instance

LogStore takes the following to be created:

Abstract Class

LogStore is an abstract class and cannot be created directly. It is created indirectly for the concrete LogStores.