[native_datafusion] [Spark SQL Tests] Missing files error handling differs from Spark

[andygrove]

Summary

2 Spark SQL tests fail because native_datafusion produces different error behavior when files are missing or mutated.

Failing Tests

• FileBasedDataSourceSuite: "Enabling/disabling ignoreMissingFiles using parquet" — native_datafusion throws "Object at location ... not found" instead of being silently ignored
• SimpleSQLViewSuite: "alter temporary view should follow current storeAnalyzedPlanForView config" — file deleted after plan creation causes native error

Root Cause

nativeDataFusionScan already has a fallback check for ignoreMissingFiles, but the test scenario may not be triggering it (e.g., the config may be set after plan creation, or the fallback condition isn't matching). Native DataFusion's error message format also differs from Spark's expected error handling.

Related

Discovered in CI for #3307 (enable native_datafusion in auto scan mode).

[andygrove]

Reopening. After rerunning under Spark 4.0.1 with native_datafusion in auto scan mode, the following tests still fail:

• FileBasedDataSourceSuite: "Enabling/disabling ignoreMissingFiles using parquet" — null did not equal "FAILED_READ_FILE.FILE_NOT_EXIST"; native path does not surface the expected Spark error class.
• SimpleSQLViewSuite: "alter temporary view should follow current storeAnalyzedPlanForView config" — CometQueryExecutionException: FileNotFound when the underlying file is deleted between planning and execution.
• ParquetV1QuerySuite: "Enabling/disabling ignoreCorruptFiles" — [INTERNAL_ERROR] Found unused message parameters of the error class '_LEGACY_ERROR_TEMP_2254'. (V2 variant passes.)