Skip to content
The Internals of Spark SQL
DynamicPartitionDataSingleWriter
Initializing search
spark-sql-internals
Spark SQL
Features
Query Execution
Internals
SQL
Connectors
High-Level APIs
Web UI
Demo
Misc
The Internals of Spark SQL
spark-sql-internals
Spark SQL
Features
Features
Aggregate Queries
Aggregate Queries
AggUtils
AggregationIterator
KVSorterIterator
ObjectAggregationIterator
ObjectAggregationMap
PhysicalAggregation
SortBasedAggregationIterator
TungstenAggregationIterator
UnsafeFixedWidthAggregationMap
UnsafeKVExternalSorter
Adaptive Query Execution
Adaptive Query Execution
AdaptiveExecutionContext
AQEOptimizer
AQEUtils
CostEvaluator
ShufflePartitionsUtil
SimpleCostEvaluator
Bloom Filter Join
Bloom Filter Join
BloomFilter
BloomFilterImpl
Bucketing
Bucketing
BucketSpec
Cache Serialization
Cache Serialization
CachedBatch
CachedBatchSerializer
CachedRDDBuilder
DefaultCachedBatchSerializer
SimpleMetricsCachedBatch
SimpleMetricsCachedBatchSerializer
Catalog Plugin API
Catalog Plugin API
CatalogExtension
CatalogHelper
CatalogManager
CatalogPlugin
Catalogs
CatalogV2Util
Column
DelegatingCatalogExtension
FunctionCatalog
MetadataColumn
StagingTableCatalog
SupportsNamespaces
SupportsCatalogOptions
TableCatalog
TableChange
V2TableWithV1Fallback
Columnar Execution
Columnar Execution
Common Table Expressions
Common Table Expressions
Configuration Properties
Connector API
Connector API
ApplyTransform
Batch
BatchWrite
CustomMetric
DataSourceV2Implicits
DataWriter
DataWriterFactory
Expression
InputPartition
LocalScan
OptionsHelper
PartitionReader
PartitionReaderFactory
PartitionSpecsHelper
Partitioning
Predicate
RewritableTransform
Scan
ScanBuilder
SessionConfigSupport
SimpleTableProvider
StagedTable
SupportsAtomicPartitionManagement
SupportsDelete
SupportsDeleteV2
SupportsDynamicOverwrite
SupportsMetadata
SupportsMetadataColumns
SupportsOverwrite
SupportsPartitionManagement
SupportsPushDownAggregates
SupportsPushDownFilters
SupportsPushDownRequiredColumns
SupportsPushDownV2Filters
SupportsRead
SupportsReportOrdering
SupportsReportPartitioning
SupportsReportStatistics
SupportsRowLevelOperations
SupportsStreamingUpdate
SupportsTruncate
SupportsWrite
Table
TableCapability
TableHelper
TableProvider
Transform
TransformHelper
TruncatableTable
V1Scan
V1Table
V1WriteBuilder
Write
WriteBuilder
Connector Expressions
Connector Expressions
Aggregation
SortOrder
Cost-Based Optimization
Cost-Based Optimization
BasicStatsPlanVisitor
CatalogColumnStat
ColumnStat
CommandUtils
EstimationUtils
JoinEstimation
LogicalPlanStats
LogicalPlanVisitor
SizeInBytesOnlyStatsPlanVisitor
Statistics
Default Columns
Default Columns
Direct Queries on Files
Direct Queries on Files
Dynamic Partition Pruning
Dynamic Partition Pruning
File-Based Data Scanning
File-Based Data Scanning
Generated Columns
Generated Columns
GeneratedColumn
Hidden File Metadata
Hidden File Metadata
MetadataAttribute
Hints (SQL)
Hints (SQL)
HintErrorHandler
HintInfo
JoinHint
JoinStrategyHint
Join Strategy Hints
Join Queries
Join Queries
Joins
Broadcast Joins
Logging
Metadata Columns
Metadata Columns
FileSourceConstantMetadataAttribute
FileSourceGeneratedMetadataAttribute
FileSourceMetadataAttribute
MetadataColumnHelper
MetadataColumnsHelper
Partition File Metadata Caching
Partition File Metadata Caching
Spark Connect
Spark Connect
CommandPlugin
ExpressionPlugin
RelationPlugin
SimpleSparkConnectService
SparkConnectAnalyzeHandler
SparkConnectInterceptorRegistry
SparkConnectPlanner
SparkConnectPlugin
SparkConnectServer
SparkConnectService
SparkConnectStreamHandler
Runtime Filtering
Runtime Filtering
Spark Thrift Server
Spark Thrift Server
SparkSQLEnv
Statistics
Subexpression Elimination
Subexpression Elimination
EquivalentExpressions
Subqueries
Subqueries
Table-Valued Functions
Table-Valued Functions
Time Travel
Time Travel
TimeTravelSpec
Transactional Writes
Transactional Writes
SQLHadoopMapReduceCommitProtocol
User-Defined Functions
User-Defined Functions
UDFRegistration
UserDefinedPythonFunction
Vectorized Decoding
Vectorized Decoding
ColumnVector
OffHeapColumnVector
OnHeapColumnVector
WritableColumnVector
ANSI Intervals
Catalog Plugin API and Multi-Catalog Support
Explaining Query Plans Improved
Observable Metrics
Hive Integration
Dynamic Partition Inserts
Vectorized Query Execution
Vectorized Query Execution
ColumnarBatch
Whole-Stage Code Generation
Whole-Stage Code Generation
Block
BufferedRowIterator
CodeGenerator
CodegenContext
GenerateColumnAccessor
GenerateMutableProjection
GenerateOrdering
GeneratePredicate
GenerateSafeProjection
GenerateUnsafeProjection
Catalyst DSL
Catalyst DSL
DslLogicalPlan
Variable Substitution
Query Execution
Query Execution
Catalyst
Catalyst
GenericStrategy
Optimizer
PlanChangeLogger
QueryPlan
QueryPlanner
Rule
RuleExecutors
TreeNode
TreePattern
TreePatternBits
Catalyst Expressions
Catalyst Expressions
AggregateExpression
AggregateFunction
AggregateWindowFunction
Aggregator
ArrayFilter
AttributeSeq
Attribute
BasePredicate
BinaryComparison
BinaryOperator
BloomFilterAggregate
BloomFilterMightContain
BoundReference
CallMethodViaReflection
CodeGeneratorWithInterpretedFallback
CodegenFallback
Collect
CollectSet
Count
CreateNamedStruct
CreateStruct
CumeDist
DeclarativeAggregate
DecodeUsingSerializer
DynamicPruningExpression
DynamicPruningSubquery
EncodeUsingSerializer
EqualNullSafe
EqualTo
ExecSubqueryExpression
Exists
ExpectsInputTypes
ExplodeBase
Expression
First
Generator
HashExpression
HashPartitioning
Higher-Order Functions
Higher-Order Functions
HigherOrderFunction
ArrayBasedSimpleHigherOrderFunction
MapBasedSimpleHigherOrderFunction
SimpleHigherOrderFunction
ImperativeAggregate
In
Inline
InSet
InSubquery
InSubqueryExec
InterpretedProjection
JsonToStructs
LessThanOrEqual
ListQuery
Literal
MonotonicallyIncreasingID
Murmur3Hash
MutableProjection
NamedExpression
Nondeterministic
OffsetWindowFunction
ParseToDate
ParseToTimestamp
PlanExpression
Predicate
Projection
PythonUDF
RowOrdering
RuntimeReplaceable
ScalaAggregator
ScalarSubquery
ScalarSubquery (ExecSubqueryExpression)
ScalaUDAF
ScalaUDF
SimpleTypedAggregateExpression
SortOrder
SparkUserDefinedFunction
Stateful
StaticInvoke
SubqueryExpression
TimeWindow
TypedImperativeAggregate
UnaryExpression
Unevaluable
UnixTimestamp
UnresolvedAttribute
UnresolvedFunction
UnresolvedGenerator
UnresolvedOrdinal
UnresolvedStar
UnsafeProjection
UserDefinedAggregator
User-Defined Functions
User-Defined Functions
User-Defined Functions
UserDefinedExpression
UserDefinedFunction
UDFs are Blackbox
UserDefinedAggregateFunction
WindowExpression
WindowFunction
WindowSpecDefinition
Execution Planning Strategies
Execution Planning Strategies
Aggregation
BasicOperators
DataSourceStrategy
DataSourceV2Strategy
FileSourceStrategy
InMemoryScans
JoinSelection
LogicalQueryStageStrategy
SparkStrategies
SparkStrategy
SpecialLimits
Window
WithCTEStrategy
Logical Analyzer
Logical Analysis Rules
Logical Analysis Rules
AddMetadataColumns
AliasViewChild
CTESubstitution
CleanupAliases
DataSourceAnalysis
ExtractWindowExpressions
FindDataSourceTable
LookupFunctions
PreWriteCheck
PreprocessTableCreation
ResolveAggregateFunctions
ResolveAliases
ResolveCatalogs
ResolveCoalesceHints
ResolveCreateNamedStruct
ResolveDefaultColumns
ResolveFunctions
ResolveGroupingAnalytics
ResolveInlineTables
ResolveInsertInto
ResolveJoinStrategyHints
ResolveOrdinalInOrderByAndGroupBy
ResolveReferences
ResolveRelations
ResolveSQLOnFile
ResolveSessionCatalog
ResolveSubquery
ResolveWindowFrame
ResolveWithCTE
RewriteDeleteFromTable
RewriteRowLevelCommand
TableCapabilityCheck
WindowFrameCoercion
WindowsSubstitution
Logical Operators
Logical Operators
AddColumns Logical Operator
Aggregate Logical Operator
AlterTable Logical Command
AlterTableAddColumnsCommand Logical Runnable Command
AlterTableCommand
AnalyzeColumn Logical Command
AnalyzeColumnCommand Logical Command
AnalyzePartitionCommand Logical Command
AnalyzeTable
AnalyzeTableCommand Logical Command
AnalyzeTablesCommand Logical Command
AppendData Logical Command
ArrowEvalPython
BaseEvalPython
CTERelationDef Unary Logical Operator
CTERelationRef Leaf Logical Operator
CacheTableCommand Runnable Logical Command
ClearCacheCommand Logical Command
CollectMetrics Logical Operator
Command — Eagerly-Executed Logical Operators
CommentOnTable Logical Command
CreateDataSourceTableAsSelectCommand Logical Command
CreateDataSourceTableCommand Logical Command
CreateTable Logical Operator
CreateTableAsSelect Logical Command
CreateTempViewUsing Logical Command
CreateViewCommand Logical Command
DataSourceV2Relation Leaf Logical Operator
DataSourceV2ScanRelation Leaf Logical Operator
DataWritingCommand Logical Commands
DeleteFromTable Logical Command
DescribeColumnCommand Logical Command
DescribeRelation Logical Command
DescribeTableCommand Logical Command
DeserializeToObject
Except Logical Operator
Expand Unary Logical Operator
ExplainCommand Logical Command
ExposesMetadataColumns
ExternalRDD Leaf Logical Operator
FlatMapGroupsWithState Logical Operator
Generate Unary Logical Operator
GlobalLimit Logical Operator
GroupingSets Unary Logical Operator
IgnoreCachedData Logical Operators
InMemoryRelation Leaf Logical Operator
InsertIntoDataSourceCommand Logical Command
InsertIntoDir Unary Logical Operator
InsertIntoHadoopFsRelationCommand Logical Command
InsertIntoStatement
InsertIntoTable Unary Logical Operator
Join Logical Operator
LeafNodes
LeafRunnableCommand — Leaf Logical Runnable Commands
LoadDataCommand
LocalRelation Leaf Logical Operator
LogicalPlan — Logical Relational Operators of Structured Query
LogicalQueryStage Leaf Logical Operator
LogicalRDD Leaf Logical Operator
LogicalRelation Leaf Logical Operator
MapPartitions Unary Logical Operator
MergeIntoTable Logical Command
MultiInstanceRelation Logical Operators
NamedRelation Logical Operators
OverwriteByExpression Logical Command
OverwritePartitionsDynamic Logical Command
ParsedStatement Logical Operators
Pivot Unary Logical Operator
Project Unary Logical Operator
RebalancePartitions Unary Logical Operator
RelationTimeTravel
Repartition Logical Operator
RepartitionByExpression Logical Operator
RepartitionOperation Unary Logical Operators
ReplaceData Logical Operator
ResolvedHint Unary Logical Operator
ResolvedTable Leaf Logical Operator
RowLevelWrite Logical Operators
RunnableCommand Logical Operators
SaveIntoDataSourceCommand Logical Command
SetCatalogAndNamespace Logical Command
ShowCreateTable Logical Command
ShowCreateTableCommand Logical Command
ShowTableProperties Logical Command
ShowTablePropertiesCommand Logical Command
ShowTables Logical Command
Sort Unary Logical Operator
SubqueryAlias Unary Logical Operator
SupportsSubquery Logical Operators
TruncateTableCommand Logical Command
UnresolvedHaving Unary Logical Operator
UnresolvedHint Unary Logical Operator
UnresolvedRelation
UnresolvedTable Leaf Logical Operator
UnresolvedTableOrView Leaf Logical Operator
UnresolvedTableValuedFunction Logical Operator
UnresolvedWith Unary Logical Operator
UpdateTable Logical Operator
V2CreateTablePlan Logical Operators
V2WriteCommand Logical Commands
View Unary Logical Operator
Window Unary Logical Operator
WithCTE Logical Operator
WithWindowDefinition Unary Logical Operator
WriteDelta Logical Operator
WriteFiles Unary Logical Operator
Logical Optimizations
Logical Optimizations
AQEPropagateEmptyRelation Adaptive Logical Optimization
CleanupDynamicPruningFilters Logical Optimization
CollapseWindow Logical Optimization
ColumnPruning Logical Optimization
CombineTypedFilters Logical Optimization
CombineUnions Logical Optimization
ComputeCurrentTime Logical Optimization
ConstantFolding Logical Optimization
ConvertToLocalRelation Logical Optimization
CostBasedJoinReorder Logical Optimization -- Join Reordering in Cost-Based Optimization
DecimalAggregates Logical Optimization
DynamicJoinSelection Adaptive Logical Optimization
EliminateResolvedHint Logical Optimization
EliminateSerialization Logical Optimization
EliminateSubqueryAliases Logical Optimization
EliminateView Logical Optimization
ExtractPythonUDFFromAggregate Logical Optimization
ExtractPythonUDFs
GetCurrentDatabase Logical Optimization
GroupBasedRowLevelOperationScanPlanning Logical Optimization
InferFiltersFromConstraints Logical Optimization Rule
InjectRuntimeFilter Logical Optimization
InlineCTE Logical Optimization
LimitPushDown Logical Optimization
NullPropagation Logical Optimization -- Nullability (NULL Value) Propagation
OptimizeIn Logical Optimization
OptimizeMetadataOnlyQuery Logical Optimization
OptimizeSubqueries Logical Optimization
PartitionPruning Logical Optimization
PropagateEmptyRelation Logical Optimization
PruneFileSourcePartitions Logical Optimization
PruneFilters
PruneHiveTablePartitions Logical Optimization
PullupCorrelatedPredicates Logical Optimization
PushDownLeftSemiAntiJoin
PushDownOperatorsToDataSource Logical Optimization
PushDownPredicate Logical Optimization
PushDownPredicates Logical Optimization
PushPredicateThroughJoin Logical Optimization
ReorderJoin Logical Optimization -- Reordering Inner and Cross Joins
ReplaceExceptWithAntiJoin Logical Optimization Rule -- Rewriting Except (DISTINCT) Operators
ReplaceExceptWithFilter Logical Optimization Rule -- Rewriting Except (DISTINCT) Operators
ReplaceExpressions Logical Optimization
RewriteCorrelatedScalarSubquery Logical Optimization
RewriteExceptAll Logical Optimization Rule -- Rewriting Except (ALL) Operators
RewritePredicateSubquery Logical Optimization
SchemaPruning Logical Optimization
SimplifyCasts Logical Optimization
UpdateAttributeNullability Logical Optimization
UpdateCTERelationStats Logical Optimization
V2ScanRelationPushDown Logical Optimization
V2Writes Logical Optimization
Physical Operators
Physical Operators
AdaptiveSparkPlanExec
AggregateCodegenSupport
AliasAwareOutputExpression
AliasAwareQueryOutputOrdering
AlterTableExec
AQEShuffleReadExec
AtomicTableWriteExec
BaseAggregateExec
BaseJoinExec
BaseSubqueryExec
BatchWriteHelper
BatchScanExec
BroadcastExchangeExec
BroadcastExchangeLike
BroadcastHashJoinExec
BroadcastNestedLoopJoinExec
BroadcastQueryStageExec
CoalesceExec
CollectLimitExec
CollectMetricsExec
ColumnarToRowExec
ColumnarToRowTransition
CreateTableAsSelectExec
CodegenSupport
DataSourceScanExec
DataSourceV2ScanExecBase
DataWritingCommandExec
DebugExec
DeleteFromTableExec
DescribeTableExec
DeserializeToObjectExec
DropNamespaceExec
EvalPythonExec
Exchange
ExecutedCommandExec
ExternalRDDScanExec
FileSourceScanExec
FilterExec
GenerateExec
HashAggregateExec
HashedRelation
HashJoin
InMemoryTableScanExec
InputAdapter
JoinCodegenSupport
LocalTableScanExec
LongHashedRelation
ObjectConsumerExec
ObjectHashAggregateExec
ObjectProducerExec
OrderPreservingUnaryExecNode
OverwriteByExpressionExec
PartitioningPreservingUnaryExecNode
ProjectExec
QueryStageExec
RangeExec
ReusedExchangeExec
ReusedSubqueryExec
RowDataSourceScanExec
RowToColumnarExec
SerializeFromObjectExec
SetCatalogAndNamespaceExec
ShowCreateTableExec
ShowTablesExec
ShowTablePropertiesExec
ShuffleExchangeExec
ShuffleExchangeLike
ShuffledHashJoinExec
ShuffledJoin
ShuffleOrigin
ShuffleQueryStageExec
SortAggregateExec
SortMergeJoinExec
SortMergeJoinExec
SortMergeJoinExec
SortMergeJoinScanner
SortExec
SparkPlan
SubqueryExec
TableWriteExecHelper
TruncateTableExec
UnaryExecNode
V2CommandExec
V2ExistingTableWriteExec
V2TableWriteExec
WholeStageCodegenExec
WindowExec
WindowExec
WindowExec
WindowExecBase
WriteFilesExec
Distribution and Partitioning
Distribution and Partitioning
Distribution
Partitioning
Distribution Specifications
Distribution Specifications
AllTuples
BroadcastDistribution
ClusteredDistribution
HashClusteredDistribution
OrderedDistribution
UnspecifiedDistribution
Broadcast Modes
Broadcast Modes
BroadcastMode
HashedRelationBroadcastMode
ShuffleSpec
Physical Optimizations
Physical Optimizations
AQEShuffleReadRule
AdjustShuffleExchangePosition
ApplyColumnarRulesAndInsertTransitions
CoalesceBucketsInJoin
CoalesceShufflePartitions
CollapseCodegenStages
DisableUnnecessaryBucketedScan
EnsureRequirements
InsertAdaptiveSparkPlan
OptimizeShuffleWithLocalRead
OptimizeSkewInRebalancePartitions
OptimizeSkewedJoin
PlanAdaptiveDynamicPruningFilters
PlanAdaptiveSubqueries
PlanDynamicPruningFilters
PlanSubqueries
RemoveRedundantProjects
RemoveRedundantSorts
ReplaceHashWithSortAgg
ReuseAdaptiveSubquery
ReuseExchange
ReuseExchangeAndSubquery
ReuseSubquery
ValidateSparkPlan
QueryExecution
QueryPlanningTracker
SparkOptimizer
SparkPlanner
Internals
Internals
Overview
DataSource
Developer API
DataSource V1 API
DataSource V1 API
DataSourceRegister
CreatableRelationProvider
RelationProvider
SchemaRelationProvider
BaseRelation
FileRelation
InsertableRelation
PrunedFilteredScan
PrunedScan
TableScan
Filter
ExecutionListenerBus
ExecutionListenerManager
SharedState
SQLConf
SQLConfHelper
StaticSQLConf
SparkSession Registries
SparkSession Registries
Catalog
Catalog
Catalog
CatalogImpl
CatalogStatistics
CatalogUtils
ExperimentalMethods
ExternalCatalog
ExternalCatalog
ExternalCatalog
InMemoryCatalog
ExternalCatalogWithListener
FunctionRegistry
FunctionRegistry
FunctionRegistry
FunctionRegistryBase
SimpleFunctionRegistry
SimpleFunctionRegistryBase
SimpleTableFunctionRegistry
TableFunctionRegistry
GlobalTempViewManager
SessionCatalog
SessionCatalog
SessionCatalog
CatalogStorageFormat
CatalogTable
CatalogTablePartition
V2SessionCatalog
SessionState
SessionState
SessionState
BaseSessionStateBuilder
SessionStateBuilder
CacheManager
CacheManager
CacheManager
RuntimeConfig
Encoder
Encoder
Encoder
ExpressionEncoder
RowEncoder
ScalaReflection
SQLExecution
SQLMetric
Tungsten Execution Backend
Tungsten Execution Backend
CatalystSerde
ExternalAppendOnlyUnsafeRowArray
HashMapGenerator
InternalRow
UnsafeHashedRelation
UnsafeRow
UnsafeRowSerializerInstance
RDDs
RDDs
DataSourceRDD
DataSourceRDDPartition
FileScanRDD
ShuffledRowRDD
SQLExecutionRDD
SQL
SQL
AbstractSqlParser
AstBuilder
CatalystSqlParser
ParserInterface
SparkSqlParser
SparkSqlAstBuilder
VariableSubstitution
Connectors
Connectors
Avro
Avro
AvroOptions
AvroFileFormat
CatalystDataToAvro
AvroDataToCatalyst
Files
Files
AggregatePushDownUtils
BasicWriteJobStatsTracker
BasicWriteTaskStats
CompressionCodecs
FileBatchWrite
FileDataSourceV2
FileFormat
FileFormatDataWriter
FileFormatWriter
FileIndex
FileIndex
FileIndex
CatalogFileIndex
InMemoryFileIndex
PartitioningAwareFileIndex
PrunedInMemoryFileIndex
FilePartition
FilePartitionReaderFactory
FileScan
FileScanBuilder
FileStatusCache
FileTable
FileWrite
FileWriterFactory
HadoopFileLinesReader
HadoopFsRelation
PartitionedFile
PartitionedFileUtil
RecordReaderIterator
SharedInMemoryCache
SchemaMergeUtils
SingleDirectoryDataWriter
WriteJobStatsTracker
WriteTaskStats
Hive
Hive
Configuration Properties
Hive Metastore
DataSinks
HiveClient
HiveClientImpl
HiveFileFormat
HiveUtils
IsolatedClientLoader
HiveTableRelation
CreateHiveTableAsSelectCommand
SaveAsHiveFile
InsertIntoHiveDirCommand
InsertIntoHiveTable
HiveTableScans
HiveTableScanExec
TableReader
HadoopTableReader
HiveSessionStateBuilder
HiveExternalCatalog
HiveSessionCatalog
HiveMetastoreCatalog
RelationConversions
ResolveHiveSerdeTable
DetermineTableStats
HiveAnalysis
JDBC
JDBC
JDBCOptions
JdbcRelationProvider
JDBCRelation
JDBCScan
JDBCScanBuilder
JDBCRDD
JdbcDialect
Kafka
Kafka
Configuration Properties
KafkaBatch
KafkaBatchWrite
KafkaBatchWriterFactory
KafkaDataConsumer
KafkaDataWriter
KafkaOffsetRangeLimit
KafkaOffsetReader
KafkaRelation
KafkaScan
KafkaSourceProvider
KafkaSourceRDD
KafkaSourceRDDPartition
KafkaTable
KafkaWrite
KafkaWriter
KafkaWriteTask
Misc
Misc
ConsumerStrategy
InternalKafkaConsumer
InternalKafkaProducerPool
JsonUtils
KafkaRowWriter
KafkaRecordToRowConverter
Options
Noop
Noop
NoopDataSource
NoopTable
NoopWriteBuilder
NoopBatchWrite
NoopStreamingWrite
Parquet
Parquet
ParquetDataSourceV2
ParquetFileFormat
ParquetFilters
ParquetOptions
ParquetPartitionReaderFactory
ParquetReadSupport
ParquetScan
ParquetScanBuilder
ParquetTable
ParquetUtils
ParquetWrite
ParquetWriteSupport
SparkToParquetSchemaConverter
SpecificParquetRecordReaderBase — Hadoop RecordReader
VectorizedColumnReader
VectorizedParquetRecordReader
BaseDynamicPartitionDataWriter
BasicWriteTaskStatsTracker
DataWritingSparkTask
DataSourceV2Utils
DynamicPartitionDataConcurrentWriter
DynamicPartitionDataSingleWriter
OutputWriter
WriteTaskStatsTracker
High-Level APIs
High-Level APIs
Column
ColumnarRule
Data Types
Data Types
AbstractDataType
ArrayType
AtomicType
CalendarInterval
DataType
Metadata
MetadataBuilder
StructField
StructType
UserDefinedType
Dataset
DataFrame
DataFrameReader
DataFrameWriter
DataFrameWriterV2
Encoders
KeyValueGroupedDataset
Observation
QueryExecutionListener
RelationalGroupedDataset
SparkSession
SparkSession.Builder
SparkSessionExtensions
Standard Functions
Standard Functions
Standard Aggregate Functions
Standard Collection Functions
Date and Time Functions
Regular Functions (Non-Aggregate Functions)
Standard Functions for Window Aggregation (Window Functions)
TypedColumn
Window Functions
Window Functions
AggregateProcessor
RangeFrame
Window
WindowFunctionFrame
WindowSpec
Web UI
Web UI
AllExecutionsPage
ExecutionPage
SparkListenerSQLExecutionEnd
SQLAppStatusListener
SQLAppStatusStore
SQLTab
Demo
Demo
Adaptive Query Execution
Connecting Spark SQL to Hive Metastore
Mult-Dimensional Aggregations
Developing CatalogPlugin
Dynamic Partition Pruning
Hive Partitioned Parquet Table and Partition Pruning
ObjectHashAggregateExec and Sort-Based Fallback Tasks
Spilling
Using JDBC Data Source to Access PostgreSQL
Misc
Misc
AggregatingAccumulator
JoinSelectionHelper
PushDownUtils
UnsafeExternalRowSorter
BindReferences
IntervalUtils
ExplainUtils
SerializerBuildHelper
Dataset, DataFrame and RDD
Dataset and SQL
Implicits
Row
Data Source API
Data Source API
CreateTableWriter
WriteConfigMethods
Dataset API
Dataset API
Operators
Typed Transformations
Untyped Transformations
Basic Actions
Actions
DataFrameNaFunctions
DataFrameStatFunctions
Column Operators
Caching and Persistence
Caching and Persistence
Caching and Persistence
User-Friendly Names of Cached Queries in web UI
Checkpointing
Performance Tuning and Debugging
Performance Tuning and Debugging
Debugging Query Execution
Performance Tuning
Case Study
CheckAnalysis
CatalystTypeConverters
SubExprUtils
PredicateHelper
ExtractEquiJoinKeys
ExtractSingleColumnNullAwareAntiJoin
ExtractJoinWithBuckets
PhysicalOperation
KnownSizeEstimation
Spark SQL
Connectors
DynamicPartitionDataSingleWriter
¶
DynamicPartitionDataSingleWriter
is...FIXME
Back to top