Skip to content
The Internals of Delta Lake
SparkMicroBatchStream
Initializing search
GitHub
Delta Lake
Features
Internals
Developer API
Delta SQL
Commands
Demos
Misc
Monitoring
Delta Kernel
Contenders
The Internals of Delta Lake
GitHub
Delta Lake
Features
Features
Catalog-Managed Tables
Catalog-Managed Tables
CatalogOwnedTableFeature
CatalogOwnedTableUtils
Commits
v4.1.0 → v4.2.0
Unity Catalog Support
Unity Catalog Support
UCCatalogManagedClient
UCCatalogManagedCommitter
UCClient
UCClientFactory
UCCommitCoordinatorBuilder
UCCommitCoordinatorClient
UCTokenBasedRestClient
UCTokenBasedRestClientFactory
UpdateMetricsHook
Delta Spark V2 Connector
Delta Spark V2 Connector
SparkTable
Append-Only Tables
Append-Only Tables
AppendOnlyTableFeature
Auto Compaction
Auto Compaction
AutoCompact
AutoCompactBase
AutoCompactRequest
AutoCompactUtils
Change Data Feed
Change Data Feed
CDCDataSpec
CDCReader
CDCReaderImpl
CDCVersionDiffInfo
CdcAddFileIndex
ChangeDataFeedTableFeature
DeltaCDFRelation
DeltaSourceCDCSupport
IndexedChangeFileSeq
TahoeChangeFileIndex
TahoeRemoveFileIndex
CDF Table-Valued Functions
CDF Table-Valued Functions
DeltaTableValueFunction
DeltaTableValueFunctions
CHECK Constraints
CHECK Constraints
AddConstraint
AlterTableAddConstraint
AlterTableDropConstraint
CheckConstraintsTableFeature
DropConstraint
Checkpoints
Checkpoints
CheckpointHook
Checkpoints
CheckpointV2
Column Invariants
Column Invariants
Invariants
InvariantsTableFeature
Column Mapping
Column Mapping
ColumnMappingTableFeature
DeltaColumnMappingBase (DeltaColumnMapping)
DeltaColumnMappingMode
Column Statistics
Column Statistics
Configuration Properties
Configuration Properties
DeltaSQLConf
Coordinated Commits
Coordinated Commits
CommitCoordinatorBuilder
CommitCoordinatorClient
CommitCoordinatorProvider
CoordinatedCommitsPreDowngradeCommand
CoordinatedCommitsTableFeature
CoordinatedCommitsUtils
DeltaCommitFileProvider
TableCommitCoordinatorClient
Data Skipping
Data Skipping
ColumnPredicateBuilder
CountStarDeltaTable
DataFiltersBuilder
DataSkippingPredicateBuilder
DataSkippingReaderBase
DeltaDataSkippingType
DeltaScan
DeltaScanGenerator
FileSizeHistogram
OptimizeMetadataOnlyDeltaQuery
PrepareDeltaScan
PrepareDeltaScanBase
PreparedDeltaFileIndex
ReadsMetadataFields
ShowCountStarDeltaTable
StatsProvider
UsesMetadataFields
Deletion Vectors
Deletion Vectors
DeletionVectorBitmapGenerator
DeletionVectorDescriptor
DeletionVectorSet
DeletionVectorStore
DeletionVectorStoredBitmap
DeletionVectorStoreUtils
DeletionVectorsPreDowngradeCommand
DeletionVectorsTableFeature
DeletionVectorUtils
DeletionVectorWriter
DMLWithDeletionVectorsHelper
HadoopFileSystemDVStore
PreprocessTableWithDVsStrategy
RowIndexFilter
RowIndexMarkingFilters
RowIndexMarkingFiltersBuilder
ScanWithDeletionVectors
StoredBitmap
TouchedFileWithDV
Developer API
Dynamic Partition Overwrite
Dynamic Partition Overwrite
DeltaDynamicPartitionOverwriteCommand
Generated Columns
Generated Columns
GeneratedColumn
GeneratedColumnsTableFeature
Identity Columns
Identity Columns
GenerateIdentityValues
IdentityColumn
IdentityColumnsTableFeature
PartitionIdentityValueGenerator
Installation
Liquid Clustering
Clustered Tables
Liquid Clustering
ClusterByParserUtils
ClusterByPlan
ClusterBySpec
ClusteredTableUtils
ClusteringMetadataDomain
ClusteringTableFeature
JsonMetadataDomainUtils
Log Cleanup
Log Cleanup
MetadataCleanup
LIMIT Pushdown
LIMIT Pushdown
Logging
Post-Commit Hooks
Post-Commit Hooks
GenerateSymlinkManifest
PostCommitHook
UpdateCatalogBase
Row Tracking
Row Tracking
DefaultRowCommitVersion
DeltaScanWithRowTrackingEnabled
GenerateRowIDs
MaterializedRowCommitVersion
MaterializedRowTrackingColumn
MetadataStructField
RowCommitVersion
RowId
RowTracking
RowTrackingBackfillCommand
RowTrackingFeature
Spark Connector
Delta Lake on Spark SQL
Spark Connector
AdmissionLimits
DeltaDataSource
DeltaOptionParser
DeltaOptions
DeltaReadOptions
DeltaSink
DeltaSource
DeltaSourceBase
DeltaSourceMetadataEvolutionSupport
DeltaSourceMetadataTrackingLog
DeltaSourceOffset
DeltaSourceSnapshot
DeltaSourceUtils
DeltaWriteOptions
DeltaWriteOptionsImpl
Options
PersistedMetadata
SchemaReadOptions
SchemaTrackingLog
SnapshotIterator
SparkMicroBatchStream
Storage
Storage
DelegatingLogStore
HadoopFileSystemLogStore
HDFSLogStore
LogStore
LogStoreAdaptor
LogStoreProvider
S3SingleDriverLogStore
Table Constraints
Table Constraints
Check
CheckDeltaInvariant
Constraint
Constraints
DeltaInvariantChecker
DeltaInvariantCheckerExec
DeltaInvariantCheckerStrategy
Invariant
InvariantViolationException
Table Features
Table Features
DomainMetadataTableFeature
FeatureAutomaticallyEnabledByMetadata
InCommitTimestampTableFeature
LegacyWriterFeature
PreDowngradeTableFeatureCommand
ReaderWriterFeature
RemovableFeature
TableFeature
TableFeatureProtocolUtils
TableFeatureSupport
VacuumProtocolCheckTableFeature
VariantTypePreviewTableFeature
VariantTypeTableFeature
WriterFeature
Table Properties
Table Properties
DeltaConfig
DeltaConfigs
Time Travel
Time Travel
DeltaTimeTravelSpec
Universal Format (UniForm)
Universal Format (UniForm)
HudiConverter
HudiConverterHook
IcebergCompat
IcebergCompatV2
IcebergConverter
IcebergConverterHook
UniversalFormat
UniversalFormatConverter
Version Checksums
Version Checksums
ChecksumHook
RecordChecksum
WriteChecksum
Internals
Internals
AbstractDeltaCatalog
Actions
Actions
Action
AddCDCFile
AddFile
CheckpointMetadata
CheckpointOnlyAction
CommitInfo
DomainMetadata
FileAction
Metadata
Protocol
RemoveFile
SetTransaction
SidecarFile
SingleAction
ColumnMappingPreDowngradeCommand
DelayedCommitProtocol
DeltaCatalog
DeltaTableIdentifier
FileFormat
FileFormat
DeltaFileFormat
DeltaParquetFileFormat
Transaction Log (DeltaLog)
Transaction Log (DeltaLog)
DeltaLog
Operation
ReadChecksum
SnapshotManagement
SnapshotDescriptor
SnapshotStateManager
VerifyChecksum
Optimistic Transactions
Optimistic Transactions
OptimisticTransaction
OptimisticTransactionImpl
TransactionalWrite
IsolationLevel
State Snapshot
State Snapshot
Snapshot
State
InitialSnapshot
InMemoryLogReplay
PartitionFiltering
LogSegment
DeltaHistoryManager
FileNames
Spark SQL Extensions
Spark SQL Extensions
AppendDelta
CheckUnresolvedRelationTimeTravel
DeltaAnalysis
DeltaSparkSessionExtension
DeltaTableV2
DeltaUnsupportedOperationsCheck
PostHocResolveUpCast
PreprocessTableDelete
PreprocessTableMerge
PreprocessTableUpdate
PreprocessTimeTravel
ResolveDeltaPathTable
StagedDeltaTableV2
File Indices
File Indices
SupportsRowIndexFilters
TahoeFileIndex
TahoeBatchFileIndex
TahoeFileIndexWithSnapshotDescriptor
TahoeLogFileIndex
DeltaLogFileIndex
WriteIntoDeltaBuilder
ApplyV2Streaming
CheckpointFileManager
DeltaAnalysisException
DeltaV1WriteBuilder
DeltaV2Mode
SupportsPathIdentifier
TransactionHelper
Developer API
Developer API
DeltaTable
DeltaTableBuilder
DeltaColumnBuilder
DeltaOptimizeBuilder
Exceptions
Exceptions
ConcurrentAppendException
ConcurrentDeleteDeleteException
ConcurrentDeleteReadException
ConcurrentTransactionException
ConcurrentWriteException
DeltaConcurrentModificationException
DeltaErrors
DeltaErrorsBase
MetadataChangedException
ProtocolChangedException
LogStore
Delta SQL
Delta SQL
DeltaSqlAstBuilder
DeltaSqlParser
Commands
Commands
DeltaCommand
ALTER TABLE
ALTER TABLE
AlterDeltaTableCommand
ADD COLUMNS
ADD CONSTRAINT
CHANGE COLUMN
AlterTableClusterByDeltaCommand
AlterTableConstraintDeltaCommand
DROP COLUMNS
DROP CONSTRAINT
DROP FEATURE
AlterTableDropFeatureDeltaCommand
REPLACE COLUMNS
AlterTableSetLocationDeltaCommand
SET TBLPROPERTIES
UNSET TBLPROPERTIES
DropFeature
RemoveColumnMappingCommand
CLONE
CLONE
CloneConvertedSource
CloneDeltaSource
CloneParquetSource
CloneSource
CloneTableBase
CloneTableBaseUtils
CloneTableCommand
CloneTableStatement
CONVERT
CONVERT
ConvertTargetTable
ConvertToDeltaCommand
ConvertToDeltaCommandUtils
DeltaConvert Utility
FileManifest
ManualListingFileManifest
MetadataLogFileManifest
ParquetTable
CREATE TABLE
CREATE TABLE
CreateDeltaTableCommand
CreateDeltaTableLike
CREATE TABLE LIKE
CREATE TABLE LIKE
DELETE
DELETE
DeleteCommand
DeleteCommandMetrics
DeltaDelete
DESCRIBE DETAIL
DESCRIBE DETAIL
DescribeDeltaDetailCommand
TableDetail
DESCRIBE HISTORY
DESCRIBE HISTORY
DeltaHistory
DescribeDeltaHistory
DescribeDeltaHistoryCommand
GENERATE
GENERATE
DeltaGenerateCommand
MERGE
MERGE
ClassicMergeExecutor
DeduplicateCDFDeletes
DeltaMergeAction
DeltaMergeBuilder
DeltaMergeInto
DeltaMergeIntoClause
WHEN Clause
DeltaMergeIntoMatchedClause
WHEN MATCHED Clause
DeltaMergeIntoMatchedDeleteClause
WHEN MATCHED DELETE Clause
DeltaMergeIntoMatchedUpdateClause
WHEN MATCHED THEN UPDATE Clause
DeltaMergeIntoNotMatchedBySourceClause
WHEN NOT MATCHED BY SOURCE Clause
DeltaMergeIntoNotMatchedBySourceDeleteClause
WHEN NOT MATCHED BY SOURCE THEN DELETE Clause
DeltaMergeIntoNotMatchedBySourceUpdateClause
WHEN NOT MATCHED BY SOURCE THEN UPDATE Clause
DeltaMergeIntoNotMatchedClause
WHEN NOT MATCHED Clause
DeltaMergeIntoNotMatchedInsertClause
WHEN NOT MATCHED THEN INSERT Clause
DeltaMergeMatchedActionBuilder
DeltaMergeNotMatchedActionBuilder
DeltaMergeNotMatchedBySourceActionBuilder
InsertOnlyMergeExecutor
JoinedRowProcessor
MergeIntoCommand
MergeIntoCommandBase
MergeIntoMaterializeSource
MergeOutputGeneration
MergeStats
OPTIMIZE
OPTIMIZE
DeltaOptimizeContext
HilbertByteArrayIndex
HilbertClustering
HilbertLongIndex
InterleaveBits
MultiDimClustering
MultiDimClusteringFunctions
OptimizeExecutor
OptimizeMetrics
OptimizeStats
OptimizeTableCommand
OptimizeTableCommandBase
PartitionerExpr
RangePartitionId
RangePartitionIdRewrite
SpaceFillingCurveClustering
ZOrderClustering
REORG
REORG
DeltaReorgTable
DeltaReorgTableCommand
ReorgTableForUpgradeUniformHelper
RESTORE
RESTORE
RestoreTableCommand
RestoreTableCommandBase
RestoreTableStatement
TimeTravel
SHOW COLUMNS
SHOW COLUMNS
ShowDeltaTableColumnsCommand
UPDATE
UPDATE
DeltaUpdateTable
UpdateCommand
VACUUM
VACUUM
VacuumCommand
VacuumCommandImpl
VacuumTableCommand
WriteIntoDelta
WriteIntoDeltaLike
Demos
Demos
Demo: Converting Parquet Dataset Into Delta Format
Demo: Catalog-Managed Tables with Unity Catalog
Demo: Change Data Feed
Demo: Column Mapping
Demo: Debugging Delta Lake Using IntelliJ IDEA
Demo: DeltaTable, DeltaLog And Snapshots
Demo: Data Skipping
Demo: dataChange
Demo: Generated Columns
Demo: Merge Operation
Demo: Observing Transaction Retries
Demo: Optimize
Demo: replaceWhere
Demo: Rolling Back Table Changes (Restore Command)
Demo: Schema Evolution
Demo: Stream Processing of Delta Table
Demo: Time Travel
Demo: Using Delta Lake as Streaming Sink in Structured Streaming
Demo: User Metadata for Labelling Commits
Demo: Vacuum
Misc
Misc
ColumnWithDefaultExprUtils
ConflictChecker
CurrentTransactionInfo
DeltaFileFormatWriter
DeltaFileOperations
DeltaJobStatisticsTracker
DeltaOptimizedWriterExec
DeltaRelation
DeltaTableOperations
DeltaTableUtils
DomainMetadataUtils
ImplicitMetadataOperation
JsonMetadataDomain
SchemaMergingUtils
SchemaUtils
StateCache
StateCache
StateCache
CachedDS
StatisticsCollection
Monitoring
Monitoring
DeltaProgressReporter
DeltaLogging
SQLMetricsReporting
Delta Kernel
Delta Kernel
DefaultEngine
Engine
FileSystemClient
Table
TableImpl
Contenders
Contenders
Delta Lake
Features
Spark Connector
SparkMicroBatchStream
¶
SparkMicroBatchStream
is...FIXME
Back to top