Skip to content

ManualListingFileManifest

ManualListingFileManifest is a FileManifest.

getFiles

getFiles: Iterator[SerializableFileStatus]

getFiles is part of the FileManifest abstraction.

getFiles...FIXME

close

close

close is part of the Closeable (Java) abstraction.

close...FIXME

HDFS FileStatus List Dataset

list: Dataset[SerializableFileStatus]

list creates a HDFS FileStatus dataset and marks it to be cached (once an action is executed).

Scala lazy value

list is a Scala lazy value and is initialized once when first accessed. Once computed it stays unchanged.

list is used when:

doList

doList(): Dataset[SerializableFileStatus]

doList...FIXME

doList is used when: