BucketSpec¶

BucketSpec is the bucketing specification of a table (the metadata of a bucketed table).

BucketSpec is a SQLConfHelper

Creating Instance¶

BucketSpec takes the following to be created:

• Number of buckets
• Bucketing Columns
• Sorting Columns

BucketSpec is created when:

• CatalogUtils is requested to normalizeBucketSpec
• AstBuilder is requested to parse bucketing specification
• TransformHelper is requested to convertTransforms
• HiveExternalCatalog is requested to restoreTableMetadata (and getBucketSpecFromTableProperties)
• HiveClientImpl is requested to convertHiveTableToCatalogTable
• DataFrameWriter is requested for the BucketSpec
• ShowCreateTableExec is requested to showTablePartitioning

Number of Buckets¶

BucketSpec is given the number of buckets when created.

The number of buckets has to be between 0 and spark.sql.sources.bucketing.maxBuckets (inclusive) or an AnalysisException is reported:

Number of buckets should be greater than 0 but less than or equal to bucketing.maxBuckets (`[bucketingMaxBuckets]`). Got `[numBuckets]`.