ConsoleSinkProvider¶

ConsoleSinkProvider is a SimpleTableProvider (Spark SQL) for console data source (based on the modern Connector API).

ConsoleSinkProvider is a DataSourceRegister (Spark SQL) and registers itself as the console data source format.

ConsoleSinkProvider is a CreatableRelationProvider (Spark SQL) (based on the legacy DataSource V1 API).

Demo¶

import org.apache.spark.sql.streaming.Trigger
val q = spark
  .readStream
  .format("rate")
  .load
  .writeStream
  .format("console") // <-- requests ConsoleSinkProvider for a sink
  .trigger(Trigger.Once)
  .start

scala> println(q.lastProgress.sink)
{
  "description" : "org.apache.spark.sql.execution.streaming.ConsoleSinkProvider@2392cfb1"
}

Short Name¶

shortName(): String

shortName returns console.

Get Table¶

getTable(
  options: CaseInsensitiveStringMap): Table

getTable returns the only available ConsoleTable object.

Create Relation¶

createRelation(
  sqlContext: SQLContext,
  mode: SaveMode,
  parameters: Map[String, String],
  data: DataFrame): BaseRelation

createRelation...FIXME