DeltaTimeTravelSpec¶
Time Travel Patterns¶
DeltaTimeTravelSpec
defines regular expressions for timestamp- and version-based time travel identifiers:
- Version URI:
(path)@[vV](some numbers)
- Timestamp URI:
(path)@(yyyyMMddHHmmssSSS)
Creating Instance¶
DeltaTimeTravelSpec
takes the following to be created:
- Timestamp
- Version
-
creationSource
identifier
DeltaTimeTravelSpec
asserts that either version or timestamp is provided (and throws an AssertionError
).
DeltaTimeTravelSpec
is created when:
DeltaTimeTravelSpec
utility is used to resolve a pathDeltaDataSource
utility is used to getTimeTravelVersion
Resolving Path¶
resolvePath(
conf: SQLConf,
identifier: String): (DeltaTimeTravelSpec, String)
resolvePath
...FIXME
resolvePath
is used when DeltaTableUtils
utility is used to extractIfPathContainsTimeTravel.
getTimestamp¶
getTimestamp(
timeZone: String): Timestamp
getTimestamp
...FIXME
getTimestamp
is used when DeltaTableUtils
utility is used to resolveTimeTravelVersion.
isApplicable¶
isApplicable(
conf: SQLConf,
identifier: String): Boolean
isApplicable
is true
when all of the following hold:
- spark.databricks.delta.timeTravel.resolveOnIdentifier.enabled is
true
- identifierContainsTimeTravel is
true
isApplicable
is used when DeltaTableUtils
utility is used to extractIfPathContainsTimeTravel.
identifierContainsTimeTravel¶
identifierContainsTimeTravel(
identifier: String): Boolean
identifierContainsTimeTravel
is true
when the given identifier
is either timestamp or version time travel pattern.