Skip to content

Action

Action is an abstraction of actions that change (the state of) a delta table.

Contract

Serializing to JSON

json: String

Serializes (converts) the (wrapped) action to JSON format

json uses Jackson library (with jackson-module-scala) as the JSON processor.

Used when:

Wrapping Up as SingleAction

wrap: SingleAction

Wraps the action into a SingleAction for serialization

Used when:

Implementations

Sealed Trait

Action is a Scala sealed trait which means that all of the implementations are in the same compilation unit (a single file).

Log Schema

Action defines logSchema that is a schema (StructType) to hold SingleActions.

import org.apache.spark.sql.delta.actions.Action.logSchema
logSchema.printTreeString
root
 |-- txn: struct (nullable = true)
 |    |-- appId: string (nullable = true)
 |    |-- version: long (nullable = false)
 |    |-- lastUpdated: long (nullable = true)
 |-- add: struct (nullable = true)
 |    |-- path: string (nullable = true)
 |    |-- partitionValues: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |    |-- size: long (nullable = false)
 |    |-- modificationTime: long (nullable = false)
 |    |-- dataChange: boolean (nullable = false)
 |    |-- stats: string (nullable = true)
 |    |-- tags: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |-- remove: struct (nullable = true)
 |    |-- path: string (nullable = true)
 |    |-- deletionTimestamp: long (nullable = true)
 |    |-- dataChange: boolean (nullable = false)
 |    |-- extendedFileMetadata: boolean (nullable = false)
 |    |-- partitionValues: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |    |-- size: long (nullable = false)
 |    |-- tags: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |-- metaData: struct (nullable = true)
 |    |-- id: string (nullable = true)
 |    |-- name: string (nullable = true)
 |    |-- description: string (nullable = true)
 |    |-- format: struct (nullable = true)
 |    |    |-- provider: string (nullable = true)
 |    |    |-- options: map (nullable = true)
 |    |    |    |-- key: string
 |    |    |    |-- value: string (valueContainsNull = true)
 |    |-- schemaString: string (nullable = true)
 |    |-- partitionColumns: array (nullable = true)
 |    |    |-- element: string (containsNull = true)
 |    |-- configuration: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |    |-- createdTime: long (nullable = true)
 |-- protocol: struct (nullable = true)
 |    |-- minReaderVersion: integer (nullable = false)
 |    |-- minWriterVersion: integer (nullable = false)
 |-- cdc: struct (nullable = true)
 |    |-- path: string (nullable = true)
 |    |-- partitionValues: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |    |-- size: long (nullable = false)
 |    |-- tags: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |-- commitInfo: struct (nullable = true)
 |    |-- version: long (nullable = true)
 |    |-- timestamp: timestamp (nullable = true)
 |    |-- userId: string (nullable = true)
 |    |-- userName: string (nullable = true)
 |    |-- operation: string (nullable = true)
 |    |-- operationParameters: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |    |-- job: struct (nullable = true)
 |    |    |-- jobId: string (nullable = true)
 |    |    |-- jobName: string (nullable = true)
 |    |    |-- runId: string (nullable = true)
 |    |    |-- jobOwnerId: string (nullable = true)
 |    |    |-- triggerType: string (nullable = true)
 |    |-- notebook: struct (nullable = true)
 |    |    |-- notebookId: string (nullable = true)
 |    |-- clusterId: string (nullable = true)
 |    |-- readVersion: long (nullable = true)
 |    |-- isolationLevel: string (nullable = true)
 |    |-- isBlindAppend: boolean (nullable = true)
 |    |-- operationMetrics: map (nullable = true)
 |    |    |-- key: string
 |    |    |-- value: string (valueContainsNull = true)
 |    |-- userMetadata: string (nullable = true)

Deserializing Action (from JSON)

fromJson(
  json: String): Action

fromJson utility...FIXME

fromJson is used when:


Last update: 2021-03-21