Table¶
Table
is an abstraction of logical structured data set:
- a directory or files on a file system
- a topic of Apache Kafka
- a table in a catalog
Table
can be loaded using TableCatalog.
Contract¶
Capabilities¶
Set<TableCapability> capabilities()
TableCapabilities of the table
Used when Table
is asked whether or not it supports a given capability
Columns¶
Column[] columns()
Columns of this table
Default: structTypeToV2Columns based on the schema
Used when Table
is asked whether or not it supports a given capability
Name¶
String name()
Name of the table
Partitioning¶
Transform[] partitioning()
Partitions of the table (as Transforms)
Default: (empty)
Used when:
- ResolveInsertInto logical analysis rule is executed
DataFrameWriter
is requested to insertInto and save- DescribeTableExec physical operator is executed
Properties¶
Map<String, String> properties()
Table properties
Default: (empty)
Used when:
- DescribeTableExec and ShowTablePropertiesExec physical operators are executed
Schema¶
StructType schema()
Deprecated
Use columns instead.
StructType of the table
Used when:
DataSourceV2Relation
utility is used to create a DataSourceV2Relation logical operatorSimpleTableProvider
is requested to inferSchema- DataSourceV2Strategy execution planning strategy is executed
- DescribeTableExec physical operator is executed
FileDataSourceV2
is requested to inferSchema- (Spark Structured Streaming)
TextSocketTable
is requested for aScanBuilder
with a read schema - (Spark Structured Streaming)
DataStreamReader
is requested to load data