StructType¶

StructType is a recursive DataType with fields and being a collection of fields itself.

StructType is used to define a schema.

Creating Instance¶

StructType takes the following to be created:

StructFields

Seq[StructField]¶

Not only does StructType has fields but is also a collection of StructFields (Seq[StructField]).

All things Seq (Scala) apply equally here.

scala> schemaTyped.foreach(println)
StructField(a,IntegerType,true)
StructField(b,StringType,true)

SQL Representation¶

StructType uses STRUCT<...> for SQL representation (in query plans or SQL statements).

Catalog Representation¶

StructType uses struct<...> for catalog representation.

Demo¶

// Generating a schema from a case class
// Because we're all properly lazy
case class Person(id: Long, name: String)
import org.apache.spark.sql.Encoders
val schema = Encoders.product[Person].schema
scala> println(schema.toDDL)
`id` BIGINT,`name` STRING

scala> schemaTyped.simpleString
res0: String = struct<a:int,b:string>

scala> schemaTyped.catalogString
res1: String = struct<a:int,b:string>

scala> schemaTyped.sql
res2: String = STRUCT<`a`: INT, `b`: STRING>