Skip to content

[fix](paimon) Preserve system table validation details - #66408

Merged
Gabriel39 merged 1 commit into
apache:masterfrom
Gabriel39:fix/doris-27700-paimon-system-table-error
Aug 4, 2026
Merged

[fix](paimon) Preserve system table validation details#66408
Gabriel39 merged 1 commit into
apache:masterfrom
Gabriel39:fix/doris-27700-paimon-system-table-error

Conversation

@Gabriel39

@Gabriel39Gabriel39 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary:

Paimon system table safety validation wrapped actionable reader-option errors with a generic connector message. As a result, SQL clients could not identify the invalid option even though the original cause contained it.

This change preserves IllegalArgumentException validation details at the SQL boundary while retaining the original cause. Other unexpected failures continue to use the generic connector message.

Release note

Paimon system table validation errors now report the invalid reader option.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
      • mvn -f fe/pom.xml -pl :fe-connector-paimon -am test -Dtest=PaimonScanPlanProviderTest#resolveSystemScanTableValidatesHiddenDataTable -DfailIfNoTests=false
      • mvn -f fe/pom.xml -pl :fe-connector-paimon -am package -Dmaven.build.cache.enabled=false -DfailIfNoTests=false
      • Paimon module: 505 tests, 0 failures, 1 skipped live-connectivity test.
      • FE Checkstyle: 0 violations.
    • Manual test
    • No need to test or manual test.

  • Behavior changed:

    • No.
    • Yes. Invalid Paimon system table reader options are visible in the SQL error message.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Gabriel39
[fix](paimon) Preserve system table validation details
fa42f94
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Paimon system table safety validation wrapped actionable option errors with a generic connector message, so SQL clients could not identify the invalid option.
### Release note
Paimon system table validation errors now report the invalid reader option.
### Check List (For Author)
- Test: Unit Test
- Targeted Paimon system table validation test
- Paimon connector package lifecycle and full module tests
- Behavior changed: Yes
- Invalid Paimon system table reader options are visible in the SQL error message
- Does this need documentation: No
@hello-stephen

hello-stephen commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Gabriel39

Gabriel39 commented Aug 4, 2026

Copy link
Copy Markdown
ContributorAuthor

run buildall

@Gabriel39

Gabriel39 commented Aug 4, 2026

Copy link
Copy Markdown
ContributorAuthor

/review

github-actions[bot]
github-actionsBot reviewed Aug 4, 2026
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review opinion: no blocking correctness issues found; this change is ready from a code-review perspective.

Critical checkpoints:

  • Goal and proof: the patch preserves the actionable Paimon reader-option validation message in a stable DorisConnectorException while retaining the original IllegalArgumentException cause. The new unit assertion is mutation-strong against the old generic message. The existing test_paimon_jni_reader_guardrails regression also checks that the system-table SQL failure contains scan.manifest.parallelism and that a safe relation override succeeds.
  • Scope and end-to-end behavior: the change is small and focused. Both scan-node property generation and split planning share resolveScanTable; the connector SPI/TCCL wrapper and Nereids/MySQL error path preserve the exception message. Checked catalog and authentication failures still use the generic contextual message and retain their causes. Normal-table validation behavior is unchanged.
  • Parallel paths and conditions: pinned system sources, authenticated fallback reload, safe relation overrides, unsupported/stale system handles, wrapper normalization, and the normal-table path were traced. No functionally parallel system-table path bypasses this boundary, and the new catch condition has a concrete user-correctable failure path.
  • Concurrency, lifecycle, configuration, compatibility, persistence, and writes: not applicable to this exception-classification-only change. It adds no shared state, threads, locks, lifecycle ownership, configuration, SPI/API or serialization surface, FE-BE value, transaction, EditLog, or data-write behavior.
  • Tests, observability, and performance: adjacent unit tests cover cause typing and the relevant option/source variants; the existing regression is the SQL-facing oracle. The more specific error improves observability without needing a log or metric, and the catch is constant-cost and off the success path. Per the review-only runner contract I did not run builds or tests locally. CheckStyle was green at review time; FE UT and compile checks were still queued.
  • User focus: no additional focus was supplied, so the entire two-file PR was reviewed without a narrower scope.

Review completion: one full convergence round completed. The two full-coverage reviewers and the separate risk-focused reviewer all returned NO_NEW_VALUABLE_FINDINGS against the same ledger and empty inline-comment set.

@hello-stephen

hello-stephen commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Aug 4, 2026
@github-actions

github-actionsBot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@hello-stephen

hello-stephen commented Aug 4, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 28855 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fa42f946451d3f6dfb6836cd6384e7dd34dde516, data reload: false
------ Round 1 ----------------------------------
============================================
q1	17639	3952	3923	3923
q2	2030	317	196	196
q3	10285	1404	800	800
q4	4683	473	346	346
q5	7484	849	584	584
q6	176	168	136	136
q7	763	786	589	589
q8	9315	1549	1534	1534
q9	5359	4033	4011	4011
q10	6768	1618	1356	1356
q11	496	356	329	329
q12	727	593	454	454
q13	18103	3285	2714	2714
q14	260	264	240	240
q15	q16	736	734	657	657
q17	1011	984	997	984
q18	6649	5644	5600	5600
q19	1172	1171	999	999
q20	831	664	576	576
q21	5935	2777	2505	2505
q22	475	364	322	322
Total cold run time: 100897 ms
Total hot run time: 28855 ms
----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4852	4760	4640	4640
q2	301	346	203	203
q3	4819	5224	4773	4773
q4	2220	2370	1411	1411
q5	4555	4599	4481	4481
q6	234	176	125	125
q7	1803	1673	1551	1551
q8	2341	2042	1996	1996
q9	7319	7129	6840	6840
q10	4294	4230	3776	3776
q11	512	367	339	339
q12	694	704	498	498
q13	2992	3269	2763	2763
q14	268	268	248	248
q15	q16	656	686	596	596
q17	1220	1204	1192	1192
q18	12145	11039	11729	11039
q19	1036	1063	1037	1037
q20	2193	2188	1907	1907
q21	5304	4505	4465	4465
q22	526	444	402	402
Total cold run time: 60284 ms
Total hot run time: 54282 ms

@hello-stephen

hello-stephen commented Aug 4, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 166660 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fa42f946451d3f6dfb6836cd6384e7dd34dde516, data reload: false
query5	4348	587	430	430
query6	466	226	206	206
query7	4865	582	344	344
query8	320	159	141	141
query9	8759	4025	4060	4025
query10	478	349	285	285
query11	5838	2200	1995	1995
query12	148	98	96	96
query13	1264	611	429	429
query14	6076	4269	3968	3968
query14_1	3807	3799	3749	3749
query15	193	191	177	177
query16	987	468	423	423
query17	891	662	551	551
query18	2454	476	322	322
query19	196	177	141	141
query20	103	106	98	98
query21	227	159	136	136
query22	13007	12946	12809	12809
query23	15824	15138	14482	14482
query23_1	14737	14732	14822	14732
query24	7442	1714	1247	1247
query24_1	1304	1257	1248	1248
query25	545	479	352	352
query26	1332	360	208	208
query27	2596	600	398	398
query28	4546	2049	2040	2040
query29	1067	608	454	454
query30	337	262	222	222
query31	1175	1112	1051	1051
query32	104	60	63	60
query33	509	310	231	231
query34	1194	1131	651	651
query35	719	754	641	641
query36	768	777	693	693
query37	148	112	92	92
query38	1832	1777	1694	1694
query39	823	834	786	786
query39_1	780	781	775	775
query40	246	157	143	143
query41	67	66	61	61
query42	95	92	91	91
query43	318	320	274	274
query44	1402	779	779	779
query45	195	171	165	165
query46	1019	1178	736	736
query47	1530	1544	1403	1403
query48	443	440	283	283
query49	585	406	302	302
query50	1088	431	341	341
query51	10448	10290	10393	10290
query52	83	86	74	74
query53	259	261	200	200
query54	280	233	214	214
query55	75	71	65	65
query56	298	288	297	288
query57	1011	996	930	930
query58	278	251	263	251
query59	1500	1560	1333	1333
query60	306	271	254	254
query61	152	145	148	145
query62	402	314	262	262
query63	235	191	197	191
query64	2857	1004	862	862
query65	3930	3841	3819	3819
query66	1857	499	376	376
query67	28437	28218	28246	28218
query68	3446	1583	1058	1058
query69	417	301	269	269
query70	889	798	791	791
query71	377	353	336	336
query72	3291	2783	2450	2450
query73	818	788	431	431
query74	4642	4530	4342	4342
query75	2438	2352	1999	1999
query76	2409	1139	772	772
query77	331	373	265	265
query78	11129	11165	10540	10540
query79	1404	1166	793	793
query80	788	546	463	463
query81	499	330	301	301
query82	646	183	140	140
query83	418	328	299	299
query84	329	160	135	135
query85	981	621	519	519
query86	371	233	220	220
query87	1968	1940	1855	1855
query88	3694	2834	2807	2807
query89	388	316	281	281
query90	1841	210	206	206
query91	212	197	172	172
query92	63	65	59	59
query93	1555	1505	1023	1023
query94	607	362	299	299
query95	771	508	488	488
query96	1089	795	379	379
query97	2451	2470	2368	2368
query98	194	191	188	188
query99	735	733	614	614
Total cold run time: 252997 ms
Total hot run time: 166660 ms

@hello-stephen

hello-stephen commented Aug 4, 2026

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 23.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fa42f946451d3f6dfb6836cd6384e7dd34dde516, data reload: false
query1	0.00	0.00	0.00
query2	0.09	0.05	0.05
query3	0.25	0.13	0.13
query4	1.60	0.14	0.13
query5	0.25	0.24	0.23
query6	1.16	0.82	0.83
query7	0.04	0.00	0.00
query8	0.06	0.04	0.04
query9	0.38	0.31	0.34
query10	0.54	0.58	0.56
query11	0.19	0.14	0.14
query12	0.18	0.13	0.13
query13	0.47	0.48	0.48
query14	1.02	0.99	1.00
query15	0.61	0.60	0.59
query16	0.31	0.30	0.31
query17	1.06	1.12	1.09
query18	0.22	0.20	0.20
query19	1.99	1.90	1.93
query20	0.02	0.01	0.01
query21	15.44	0.24	0.15
query22	4.70	0.05	0.05
query23	16.15	0.31	0.12
query24	2.98	0.44	0.33
query25	0.10	0.05	0.05
query26	0.74	0.21	0.16
query27	0.05	0.03	0.03
query28	3.49	0.82	0.34
query29	12.50	4.20	3.27
query30	0.27	0.15	0.15
query31	2.78	0.56	0.32
query32	3.22	0.58	0.50
query33	3.11	3.08	3.14
query34	15.43	3.91	3.27
query35	3.25	3.24	3.24
query36	0.54	0.43	0.44
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.15
query41	0.09	0.03	0.04
query42	0.03	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 95.7 s
Total hot run time: 23.95 s

@Gabriel39
Gabriel39 merged commit 99dd2b3 into apache:masterAug 4, 2026
31 of 32 checks passed
@HappenLeeHappenLee mentioned this pull request Aug 11, 2026
Open
@Gabriel39Gabriel39 mentioned this pull request Aug 17, 2026
Open
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Aug 17, 2026
@Gabriel39@wyxxxcat
[fix](paimon) Preserve system table validation details (apache#66408)
9dac80f
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@zhangstar333zhangstar333zhangstar333 approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/4.1.xdev/4.1.x-conflictreviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Gabriel39@hello-stephen@zhangstar333