Table¶
Table is an abstraction of logical structured data set:
- a directory or files on a file system
- a topic of Apache Kafka
- a table in a catalog
Table can be loaded using TableCatalog.
Contract¶
Capabilities¶
Set<TableCapability> capabilities()
TableCapabilities of the table
Used when Table is asked whether or not it supports a given capability
Columns¶
Column[] columns()
Columns of this table
Default: structTypeToV2Columns based on the schema
Used when Table is asked whether or not it supports a given capability
Name¶
String name()
Name of the table
Partitioning¶
Transform[] partitioning()
Partitions of the table (as Transforms)
Default: (empty)
Used when:
- ResolveInsertInto logical analysis rule is executed
DataFrameWriteris requested to insertInto and save- DescribeTableExec physical operator is executed
Properties¶
Map<String, String> properties()
Table properties
Default: (empty)
Used when:
- DescribeTableExec and ShowTablePropertiesExec physical operators are executed
Schema¶
StructType schema()
Deprecated
Use columns instead.
StructType of the table
Used when:
DataSourceV2Relationutility is used to create a DataSourceV2Relation logical operatorSimpleTableProvideris requested to inferSchema- DataSourceV2Strategy execution planning strategy is executed
- DescribeTableExec physical operator is executed
FileDataSourceV2is requested to inferSchema- (Spark Structured Streaming)
TextSocketTableis requested for aScanBuilderwith a read schema - (Spark Structured Streaming)
DataStreamReaderis requested to load data