Skip to content

branch-4.1: [chore](paimon) Upgrade Paimon to 1.4.2 - #66346

Merged
Gabriel39 merged 9 commits into
apache:branch-4.1from
suxiaogang223:codex/upgrade-paimon-1.4.2
Aug 5, 2026
Merged

branch-4.1: [chore](paimon) Upgrade Paimon to 1.4.2#66346
Gabriel39 merged 9 commits into
apache:branch-4.1from
suxiaogang223:codex/upgrade-paimon-1.4.2

Conversation

@suxiaogang223

@suxiaogang223suxiaogang223 commented Aug 1, 2026

Copy link
Copy Markdown
Member

What changes

  • Upgrade the FE Paimon dependency from 1.3.1 to 1.4.2.
  • Update the Paimon test IO manager implementation for the new pickTempDir() API.

Why

Use the latest stable Paimon release and keep the Doris Paimon connector compatible with its updated IO manager interface.

Validation

  • DISABLE_BUILD_UI=ON ./build.sh --fe
  • FE Maven reactor completed successfully with BUILD SUCCESS.
  • UI compilation was skipped.
  • The build reused the main checkout's thirdparty/installed.

@hello-stephen

hello-stephen commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
suxiaogang223 changed the base branch from master to branch-4.1August 1, 2026 06:23
@suxiaogang223
suxiaogang223 marked this pull request as ready for review August 1, 2026 06:23
@suxiaogang223

suxiaogang223 commented Aug 1, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@suxiaogang223suxiaogang223 changed the title Upgrade Paimon to 1.4.2[chore](paimon) Upgrade Paimon to 1.4.2Aug 1, 2026
@suxiaogang223
suxiaogang223 marked this pull request as draft August 1, 2026 06:27
@suxiaogang223suxiaogang223 changed the title [chore](paimon) Upgrade Paimon to 1.4.2branch-4.1: [chore](paimon) Upgrade Paimon to 1.4.2Aug 1, 2026
@suxiaogang223
suxiaogang223 marked this pull request as ready for review August 1, 2026 06:29
@Gabriel39

Gabriel39 commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

run buildall

Gabriel39
Gabriel39 requested changes Aug 1, 2026
View reviewed changes

@Gabriel39Gabriel39 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found one planning-path performance regression in the Paimon 1.4 API adaptation. Please keep the merged-row-count computation guarded by COUNT(*) pushdown and add regression coverage for non-count scans.

Comment threadfe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java Outdated
@hello-stephen

hello-stephen commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 87.50% (7/8) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/8) 🎉
Increment coverage report
Complete coverage report

@suxiaogang223

suxiaogang223 commented Aug 2, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

1 similar comment
@suxiaogang223

suxiaogang223 commented Aug 2, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (9/9) 🎉
Increment coverage report
Complete coverage report

suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Aug 4, 2026
@suxiaogang223
[fix](paimon) Adapt fallback table handling to Paimon 1.4.2
b4b70a6
### What problem does this PR solve?
Issue Number: None
Related PR: apache#66346
Problem Summary: Rebasing the Paimon 1.4.2 upgrade onto the current branch-4.1 exposed FallbackReadFileStoreTable API changes in the recently backported JNI reader guardrails. Replace the removed fallback accessor, use the new constructor, and preserve fallback or primary branch read priority when rebuilding manifest-capped planning tables.
### Release note
None
### Check List (For Author)
- Test:
- Manual test: ./build.sh --fe
- Unit Test: PaimonReaderOptionsTest, PaimonTableValuedFunctionTest, and PaimonJniScannerTest
- Behavior changed: No
- Does this need documentation: No
@suxiaogang223
suxiaogang223force-pushed the codex/upgrade-paimon-1.4.2 branch from 97d99f7 to b4b70a6CompareAugust 4, 2026 08:22
@suxiaogang223

suxiaogang223 commented Aug 4, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 9.57% (9/94) 🎉
Increment coverage report
Complete coverage report

@suxiaogang223
suxiaogang223force-pushed the codex/upgrade-paimon-1.4.2 branch from b4b70a6 to b6d3fdeCompareAugust 5, 2026 02:48
suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Aug 5, 2026
@suxiaogang223
[fix](paimon) Adapt fallback table handling to Paimon 1.4.2
0d81801
### What problem does this PR solve?
Issue Number: None
Related PR: apache#66346
Problem Summary: Rebasing the Paimon 1.4.2 upgrade onto the current branch-4.1 exposed FallbackReadFileStoreTable API changes in the recently backported JNI reader guardrails. Replace the removed fallback accessor, use the new constructor, and preserve fallback or primary branch read priority when rebuilding manifest-capped planning tables.
### Release note
None
### Check List (For Author)
- Test:
- Manual test: ./build.sh --fe
- Unit Test: PaimonReaderOptionsTest, PaimonTableValuedFunctionTest, and PaimonJniScannerTest
- Behavior changed: No
- Does this need documentation: No
@suxiaogang223

suxiaogang223 commented Aug 5, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 0.00% (0/653) 🎉
Increment coverage report
Complete coverage report

@suxiaogang223

suxiaogang223 commented Aug 5, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@hello-stephen

hello-stephen commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 88.24% (15/17) 🎉
Increment coverage report
Complete coverage report

suxiaogang223and others added 9 commits August 5, 2026 19:20
@suxiaogang223
Upgrade Paimon to 1.4.2
41d7cb8
@suxiaogang223
[fix](paimon) Adapt FE to Paimon 1.4 APIs
a83534b
@suxiaogang223
[fix](paimon) Isolate BE connector dependencies
c65df70
@suxiaogang223
[fix](paimon) Avoid merged row count for regular scans
da5f040
@suxiaogang223
[fix](paimon) Remove stale runtime classes
4abeff8
@suxiaogang223
[fix](paimon) Adapt fallback table handling to Paimon 1.4.2
b383f82
### What problem does this PR solve?
Issue Number: None
Related PR: apache#66346
Problem Summary: Rebasing the Paimon 1.4.2 upgrade onto the current branch-4.1 exposed FallbackReadFileStoreTable API changes in the recently backported JNI reader guardrails. Replace the removed fallback accessor, use the new constructor, and preserve fallback or primary branch read priority when rebuilding manifest-capped planning tables.
### Release note
None
### Check List (For Author)
- Test:
- Manual test: ./build.sh --fe
- Unit Test: PaimonReaderOptionsTest, PaimonTableValuedFunctionTest, and PaimonJniScannerTest
- Behavior changed: No
- Does this need documentation: No
@suxiaogang223
[fix](regression) Update Iceberg fixture for Paimon 1.4.2
54c4b6c
@suxiaogang223
fix(paimon): keep file IO providers in connector
592d2aa
@suxiaogang223
[fix](paimon) Add Hive connector to FE runtime
9ccdce6
@suxiaogang223
suxiaogang223force-pushed the codex/upgrade-paimon-1.4.2 branch from cc4920e to 9ccdce6CompareAugust 5, 2026 11:24
@suxiaogang223

suxiaogang223 commented Aug 5, 2026

Copy link
Copy Markdown
MemberAuthor

run buildall

@Gabriel39
Gabriel39 merged commit 776adf2 into apache:branch-4.1Aug 5, 2026
29 of 33 checks passed
@suxiaogang223suxiaogang223 mentioned this pull request Aug 6, 2026
Open
@suxiaogang223suxiaogang223 mentioned this pull request Sep 1, 2026
Open
12 tasks
suxiaogang223 added a commit to suxiaogang223/doris that referenced this pull request Sep 3, 2026
@suxiaogang223
[chore](paimon) Upgrade Paimon to 1.4.2
551f4c1
### What problem does this PR solve?
Issue Number: closeapache#65086
Related PR: apache#66346
Problem Summary: Upgrade the master Paimon runtime from 1.3.1 to 1.4.2 and adapt the JNI scanner and Connector SPI read path to the updated fallback-table, catalog, partition, scan-statistics, and split-count APIs. This establishes the dependency baseline required by the branch-4.1 table-write forward-port without importing unrelated write implementations.
### Release note
Upgrade the Paimon connector runtime to 1.4.2.
### Check List (For Author)
- Test: Unit Test
- PaimonJniScannerTest: 33 tests passed
- Connector Paimon targeted tests: 182 tests passed
- Maven validate passed
- Behavior changed: Yes (Paimon runtime is upgraded from 1.3.1 to 1.4.2)
- Does this need documentation: No
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguoleiyiguoleiyiguolei approved these changes
@Gabriel39Gabriel39Gabriel39 approved these changes

Assignees

No one assigned

Labels

dev/5.0.x

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@suxiaogang223@hello-stephen@Gabriel39@yiguolei