ResolveSQLOnFile Logical Evaluation Rule¶

ResolveSQLOnFile is an extended resolution rule for hive and non-hive sessions for Direct Queries on Files.

ResolveSQLOnFile is a logical rule.

Creating Instance¶

ResolveSQLOnFile takes the following to be created:

• SparkSession

ResolveSQLOnFile is created when:

• HiveSessionStateBuilder is requested for the Analyzer
• BaseSessionStateBuilder is requested for the Analyzer

Executing Rule¶

apply(
  plan: LogicalPlan): LogicalPlan

apply handles the following logical operators:

• RelationTimeTravel
• UnresolvedRelation

maybeSQLFile¶

maybeSQLFile(
  u: UnresolvedRelation): Boolean

maybeSQLFile holds true when the following are all true:

1. spark.sql.runSQLOnFiles is enabled
2. The given UnresolvedRelation is two-part (i.e., uses a single . to separate the data source part from the path itself, datasource`.`path).