The Internals of Delta Lake
IcebergConverter
Initializing search
    delta-lake-internals
    • Delta Lake
    • Features
    • Internals
    • Developer API
    • Delta SQL
    • Commands
    • Demos
    • Misc
    • Monitoring
    • Delta Kernel
    • Contenders
    delta-lake-internals
    • Delta Lake
    • Features
      • Append-Only Tables
        • AppendOnlyTableFeature
      • Auto Compaction
        • AutoCompact
        • AutoCompactBase
        • AutoCompactRequest
        • AutoCompactUtils
      • Change Data Feed
        • CDCDataSpec
        • CDCReader
        • CDCReaderImpl
        • CDCVersionDiffInfo
        • CdcAddFileIndex
        • ChangeDataFeedTableFeature
        • DeltaCDFRelation
        • DeltaSourceCDCSupport
        • IndexedChangeFileSeq
        • TahoeChangeFileIndex
        • TahoeRemoveFileIndex
      • CDF Table-Valued Functions
        • DeltaTableValueFunction
        • DeltaTableValueFunctions
      • CHECK Constraints
        • AddConstraint
        • AlterTableAddConstraint
        • AlterTableDropConstraint
        • CheckConstraintsTableFeature
        • DropConstraint
      • Checkpoints
        • CheckpointHook
        • CheckpointV2
        • Checkpoints
      • Column Invariants
        • Invariants
        • InvariantsTableFeature
      • Column Mapping
        • ColumnMappingTableFeature
        • DeltaColumnMappingBase (DeltaColumnMapping)
        • DeltaColumnMappingMode
      • Column Statistics
      • Configuration Properties
        • DeltaSQLConf
      • Data Skipping
        • ColumnPredicateBuilder
        • CountStarDeltaTable
        • DataFiltersBuilder
        • DataSkippingPredicateBuilder
        • DataSkippingReaderBase
        • DeltaDataSkippingType
        • DeltaScan
        • DeltaScanGenerator
        • FileSizeHistogram
        • OptimizeMetadataOnlyDeltaQuery
        • PrepareDeltaScan
        • PrepareDeltaScanBase
        • PreparedDeltaFileIndex
        • ReadsMetadataFields
        • ShowCountStarDeltaTable
        • StatsProvider
        • UsesMetadataFields
      • Deletion Vectors
        • DMLWithDeletionVectorsHelper
        • DeletionVectorBitmapGenerator
        • DeletionVectorDescriptor
        • DeletionVectorSet
        • DeletionVectorStore
        • DeletionVectorStoreUtils
        • DeletionVectorStoredBitmap
        • DeletionVectorUtils
        • DeletionVectorWriter
        • DeletionVectorsTableFeature
        • HadoopFileSystemDVStore
        • PreprocessTableWithDVsStrategy
        • RowIndexFilter
        • RowIndexMarkingFilters
        • RowIndexMarkingFiltersBuilder
        • ScanWithDeletionVectors
        • StoredBitmap
        • TouchedFileWithDV
      • Developer API
      • Dynamic Partition Overwrite
        • DeltaDynamicPartitionOverwriteCommand
      • Generated Columns
        • GeneratedColumn
        • GeneratedColumnsTableFeature
      • Identity Columns
        • GenerateIdentityValues
        • IdentityColumn
        • IdentityColumnsTableFeature
        • PartitionIdentityValueGenerator
      • Installation
      • Liquid Clustering
        Clustered Tables
        • ClusterByParserUtils
        • ClusterByPlan
        • ClusterBySpec
        • ClusteredTableUtils
        • ClusteringMetadataDomain
        • ClusteringTableFeature
        • JsonMetadataDomainUtils
      • Log Cleanup
        • MetadataCleanup
      • LIMIT Pushdown
      • Logging
      • Post-Commit Hooks
        • GenerateSymlinkManifest
        • PostCommitHook
        • UpdateCatalogBase
      • Row Tracking
        • DefaultRowCommitVersion
        • DeltaScanWithRowTrackingEnabled
        • GenerateRowIDs
        • MaterializedRowCommitVersion
        • MaterializedRowTrackingColumn
        • MetadataStructField
        • RowCommitVersion
        • RowId
        • RowTracking
        • RowTrackingFeature
      • Spark Connector
        Delta Lake on Spark SQL
        • AdmissionLimits
        • DeltaDataSource
        • DeltaOptionParser
        • DeltaOptions
        • DeltaReadOptions
        • DeltaSink
        • DeltaSource
        • DeltaSourceBase
        • DeltaSourceMetadataEvolutionSupport
        • DeltaSourceMetadataTrackingLog
        • DeltaSourceOffset
        • DeltaSourceSnapshot
        • DeltaSourceUtils
        • DeltaWriteOptions
        • DeltaWriteOptionsImpl
        • Options
        • SchemaTrackingLog
        • SnapshotIterator
      • Storage
        • DelegatingLogStore
        • HDFSLogStore
        • HadoopFileSystemLogStore
        • LogStore
        • LogStoreAdaptor
        • LogStoreProvider
        • S3SingleDriverLogStore
      • Table Constraints
        • Check
        • CheckDeltaInvariant
        • Constraint
        • Constraints
        • DeltaInvariantChecker
        • DeltaInvariantCheckerExec
        • DeltaInvariantCheckerStrategy
        • Invariant
        • InvariantViolationException
      • Table Features
        • DomainMetadataTableFeature
        • FeatureAutomaticallyEnabledByMetadata
        • LegacyWriterFeature
        • ReaderWriterFeature
        • TableFeature
        • TableFeatureProtocolUtils
        • TableFeatureSupport
        • WriterFeature
      • Table Properties
        • DeltaConfig
        • DeltaConfigs
      • Time Travel
        • DeltaTimeTravelSpec
      • Universal Format (UniForm)
        • HudiConverter
        • HudiConverterHook
        • IcebergCompat
        • IcebergCompatV2
        • IcebergConverter
        • IcebergConverterHook
        • UniversalFormat
        • UniversalFormatConverter
        • Action
        • AddCDCFile
        • AddFile
        • CommitInfo
        • DomainMetadata
        • FileAction
        • Metadata
        • Protocol
        • RemoveFile
        • SetTransaction
        • SingleAction
      • DelayedCommitProtocol
      • DeltaErrorsBase
      • DeltaTableIdentifier
        • DeltaFileFormat
        • DeltaParquetFileFormat
        • DeltaLog
        • Operation
        • ReadChecksum
        • SnapshotManagement
        • SnapshotDescriptor
        • SnapshotStateManager
        • VerifyChecksum
        • OptimisticTransaction
        • OptimisticTransactionImpl
        • TransactionalWrite
        • IsolationLevel
        • Snapshot
        • State
        • InitialSnapshot
        • InMemoryLogReplay
        • PartitionFiltering
        • LogSegment
      • DeltaHistoryManager
      • FileNames
        • AppendDelta
        • CheckUnresolvedRelationTimeTravel
        • DeltaAnalysis
        • DeltaCatalog
        • DeltaSparkSessionExtension
        • DeltaTableV2
        • DeltaUnsupportedOperationsCheck
        • PostHocResolveUpCast
        • PreprocessTableDelete
        • PreprocessTableMerge
        • PreprocessTableUpdate
        • PreprocessTimeTravel
        • ResolveDeltaPathTable
        • StagedDeltaTableV2
        • SupportsRowIndexFilters
        • TahoeFileIndex
        • TahoeBatchFileIndex
        • TahoeFileIndexWithSnapshotDescriptor
        • TahoeLogFileIndex
        • DeltaLogFileIndex
      • WriteIntoDeltaBuilder
      • DeltaTable
      • DeltaTableBuilder
      • DeltaColumnBuilder
      • DeltaOptimizeBuilder
      • Exceptions
        • ConcurrentAppendException
        • ConcurrentDeleteDeleteException
        • ConcurrentDeleteReadException
        • ConcurrentTransactionException
        • ConcurrentWriteException
        • DeltaConcurrentModificationException
        • DeltaErrors
        • MetadataChangedException
        • ProtocolChangedException
      • LogStore
    • Delta SQL
      • DeltaSqlAstBuilder
      • DeltaSqlParser
    • Commands
      • DeltaCommand
      • ALTER TABLE
        • AlterDeltaTableCommand
        • ADD COLUMNS
        • ADD CONSTRAINT
        • CHANGE COLUMN
        • DROP COLUMNS
        • DROP CONSTRAINT
        • DROP FEATURE
        • REPLACE COLUMNS
        • AlterTableSetLocationDeltaCommand
        • SET TBLPROPERTIES
        • UNSET TBLPROPERTIES
        • RemoveColumnMappingCommand
      • CLONE
        • CloneConvertedSource
        • CloneDeltaSource
        • CloneParquetSource
        • CloneSource
        • CloneTableBase
        • CloneTableBaseUtils
        • CloneTableCommand
        • CloneTableStatement
      • CONVERT
        • ConvertTargetTable
        • ConvertToDeltaCommand
        • ConvertToDeltaCommandUtils
        • DeltaConvert Utility
        • FileManifest
        • ManualListingFileManifest
        • MetadataLogFileManifest
        • ParquetTable
      • CREATE TABLE
        • CreateDeltaTableCommand
      • CREATE TABLE LIKE
      • DELETE
        • DeleteCommand
        • DeleteCommandMetrics
        • DeltaDelete
      • DESCRIBE DETAIL
        • DescribeDeltaDetailCommand
        • TableDetail
      • DESCRIBE HISTORY
        • DeltaHistory
        • DescribeDeltaHistory
        • DescribeDeltaHistoryCommand
      • GENERATE
        • DeltaGenerateCommand
      • MERGE
        • ClassicMergeExecutor
        • DeduplicateCDFDeletes
        • DeltaMergeAction
        • DeltaMergeBuilder
        • DeltaMergeInto
        • DeltaMergeIntoClause
          WHEN Clause
        • DeltaMergeIntoMatchedClause
          WHEN MATCHED Clause
        • DeltaMergeIntoMatchedDeleteClause
          WHEN MATCHED DELETE Clause
        • DeltaMergeIntoMatchedUpdateClause
          WHEN MATCHED THEN UPDATE Clause
        • DeltaMergeIntoNotMatchedBySourceClause
          WHEN NOT MATCHED BY SOURCE Clause
        • DeltaMergeIntoNotMatchedBySourceDeleteClause
          WHEN NOT MATCHED BY SOURCE THEN DELETE Clause
        • DeltaMergeIntoNotMatchedBySourceUpdateClause
          WHEN NOT MATCHED BY SOURCE THEN UPDATE Clause
        • DeltaMergeIntoNotMatchedClause
          WHEN NOT MATCHED Clause
        • DeltaMergeIntoNotMatchedInsertClause
          WHEN NOT MATCHED THEN INSERT Clause
        • DeltaMergeMatchedActionBuilder
        • DeltaMergeNotMatchedActionBuilder
        • DeltaMergeNotMatchedBySourceActionBuilder
        • InsertOnlyMergeExecutor
        • JoinedRowProcessor
        • MergeIntoCommand
        • MergeIntoCommandBase
        • MergeIntoMaterializeSource
        • MergeOutputGeneration
        • MergeStats
      • OPTIMIZE
        • DeltaOptimizeContext
        • HilbertByteArrayIndex
        • HilbertClustering
        • HilbertLongIndex
        • InterleaveBits
        • MultiDimClustering
        • MultiDimClusteringFunctions
        • OptimizeExecutor
        • OptimizeMetrics
        • OptimizeStats
        • OptimizeTableCommand
        • OptimizeTableCommandBase
        • PartitionerExpr
        • RangePartitionId
        • RangePartitionIdRewrite
        • SpaceFillingCurveClustering
        • ZOrderClustering
      • REORG
        • DeltaReorgTable
        • DeltaReorgTableCommand
        • ReorgTableForUpgradeUniformHelper
      • RESTORE
        • RestoreTableCommand
        • RestoreTableCommandBase
        • RestoreTableStatement
        • TimeTravel
      • SHOW COLUMNS
        • ShowDeltaTableColumnsCommand
      • UPDATE
        • DeltaUpdateTable
        • UpdateCommand
      • VACUUM
        • VacuumCommand
        • VacuumCommandImpl
        • VacuumTableCommand
      • WriteIntoDelta
      • WriteIntoDeltaLike
    • Demos
      • Change Data Feed
      • Generated Columns
      • Column Mapping
      • Rolling Back Table Changes (Restore Command)
      • Optimize
      • Data Skipping
      • Time Travel
      • Vacuum
      • dataChange
      • replaceWhere
      • Merge Operation
      • Converting Parquet Dataset Into Delta Format
      • Stream Processing of Delta Table
      • Using Delta Lake as Streaming Sink in Structured Streaming
      • Debugging Delta Lake Using IntelliJ IDEA
      • Observing Transaction Retries
      • DeltaTable, DeltaLog And Snapshots
      • Schema Evolution
      • User Metadata for Labelling Commits
      • ColumnWithDefaultExprUtils
      • ConflictChecker
      • CurrentTransactionInfo
      • DeltaFileFormatWriter
      • DeltaFileOperations
      • DeltaJobStatisticsTracker
      • DeltaOptimizedWriterExec
      • DeltaRelation
      • DeltaTableOperations
      • DeltaTableUtils
      • DomainMetadataUtils
      • ImplicitMetadataOperation
      • JsonMetadataDomain
      • SchemaMergingUtils
      • SchemaUtils
        • StateCache
        • CachedDS
      • StatisticsCollection
      • DeltaProgressReporter
      • DeltaLogging
      • SQLMetricsReporting
    • Delta Kernel
      • DefaultEngine
      • Engine
      • FileSystemClient
      • Table
      • TableImpl
    • Contenders
    1. Delta Lake
    2. Features
    3. Universal Format (UniForm)

    IcebergConverter

    Copyright © 2024-2025 Jacek Laskowski
    Made with Material for MkDocs Insiders