DataWritingCommand -- Logical Commands That Write Query Data¶

DataWritingCommand is an <> of the <> for <> that write the result of executing <> (query data) to a relation when <>.

DataWritingCommand is resolved to a <> physical operator when BasicOperators execution planning strategy is executed (i.e. plan a <> to a <>).

[[contract]] .DataWritingCommand Contract [cols="1m,2",options="header",width="100%"] |=== | Property | Description

| outputColumnNames a| [[outputColumnNames]]

[source, scala]¶

outputColumnNames: Seq[String]¶

The output column names of the <>

Used when DataWritingCommand is requested for the <>

| query a| [[query]]

[source, scala]¶

query: LogicalPlan¶

The analyzed <> representing the data to write (i.e. whose result will be inserted into a relation)

Used when DataWritingCommand is requested for the <> and <>.

| run a| [[run]]

[source, scala]¶

run( sparkSession: SparkSession, child: SparkPlan): Seq[Row]

Executes the command to write query data (the result of executing SparkPlan.md[structured query])

Used when:

• DataWritingCommandExec physical operator is requested for the sideEffectResult
• CreateDataSourceTableAsSelectCommand logical command is executed |===

[[children]] When requested for the <>, DataWritingCommand simply returns the <>.

[[extensions]] .DataWritingCommands (Direct Implementations and Extensions Only) [cols="1,2",options="header",width="100%"] |=== | DataWritingCommand | Description

| CreateDataSourceTableAsSelectCommand.md[CreateDataSourceTableAsSelectCommand] | [[CreateDataSourceTableAsSelectCommand]]

| hive/CreateHiveTableAsSelectCommand.md[CreateHiveTableAsSelectCommand] | [[CreateHiveTableAsSelectCommand]]

| InsertIntoHadoopFsRelationCommand | [[InsertIntoHadoopFsRelationCommand]]

| hive/SaveAsHiveFile.md[SaveAsHiveFile] | [[SaveAsHiveFile]] Commands that write query result as Hive files (i.e. hive/InsertIntoHiveDirCommand.md[InsertIntoHiveDirCommand] and hive/InsertIntoHiveTable.md[InsertIntoHiveTable])

|===

=== [[basicWriteJobStatsTracker]] basicWriteJobStatsTracker Method

[source, scala]¶

basicWriteJobStatsTracker(hadoopConf: Configuration): BasicWriteJobStatsTracker¶

basicWriteJobStatsTracker simply creates and returns a new BasicWriteJobStatsTracker (with the given Hadoop Configuration and the <>).

[NOTE]¶

basicWriteJobStatsTracker is used when:

• SaveAsHiveFile is requested to <> (when hive/InsertIntoHiveDirCommand.md[InsertIntoHiveDirCommand] and hive/InsertIntoHiveTable.md[InsertIntoHiveTable] logical commands are executed)

* InsertIntoHadoopFsRelationCommand logical command is executed¶

=== [[outputColumns]] Output Columns -- outputColumns Method

[source, scala]¶

outputColumns: Seq[Attribute]¶

outputColumns...FIXME

[NOTE]¶

outputColumns is used when:

• hive/CreateHiveTableAsSelectCommand.md[CreateHiveTableAsSelectCommand], hive/InsertIntoHiveDirCommand.md[InsertIntoHiveDirCommand] and InsertIntoHadoopFsRelationCommand logical commands are executed

* SaveAsHiveFile is requested to <>¶

Performance Metrics¶