Skip to content

[feature](Paimon) support deletion vector for Paimon naive reader - #34743

Merged
AshinGau merged 25 commits into
apache:masterfrom
suxiaogang223:paimon_deletion_vector
May 20, 2024
Merged

[feature](Paimon) support deletion vector for Paimon naive reader#34743
AshinGau merged 25 commits into
apache:masterfrom
suxiaogang223:paimon_deletion_vector

Conversation

@suxiaogang223

@suxiaogang223suxiaogang223 commented May 13, 2024•
edited
Loading

Copy link
Copy Markdown
Member

Proposed changes

support deletion vector for Paimon naive reader

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot

doris-robot commented May 13, 2024

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

AshinGau
AshinGau reviewed May 14, 2024
View reviewed changes
Comment threadbe/src/vec/exec/format/orc/vorc_reader.cpp Outdated
@suxiaogang223
suxiaogang223force-pushed the paimon_deletion_vector branch from 726486a to fcb9e1eCompareMay 15, 2024 07:34
github-actions[bot]
github-actionsBot reviewed May 15, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/vec/exec/format/table/paimon_jni_reader.h Outdated
github-actions[bot]
github-actionsBot reviewed May 16, 2024
View reviewed changes

@github-actionsgithub-actionsBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment threadbe/src/vec/exec/format/table/paimon_jni_reader.h
@github-actions

github-actionsBot commented May 16, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@suxiaogang223

suxiaogang223 commented May 16, 2024

Copy link
Copy Markdown
MemberAuthor

run buildall

@github-actions

github-actionsBot commented May 16, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented May 16, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.68% (9004/25233)
Line Coverage: 27.36% (74488/272277)
Region Coverage: 26.60% (38511/144784)
Branch Coverage: 23.43% (19645/83862)
Coverage Report: http://coverage.selectdb-in.cc/coverage/46411e10c217eb6457a62c1a38c5e62a36798404_46411e10c217eb6457a62c1a38c5e62a36798404/report/index.html

@suxiaogang223

suxiaogang223 commented May 17, 2024

Copy link
Copy Markdown
MemberAuthor

run buildall

@doris-robot

doris-robot commented May 17, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.67% (9003/25239)
Line Coverage: 27.35% (74490/272328)
Region Coverage: 26.60% (38521/144799)
Branch Coverage: 23.43% (19655/83872)
Coverage Report: http://coverage.selectdb-in.cc/coverage/46411e10c217eb6457a62c1a38c5e62a36798404_46411e10c217eb6457a62c1a38c5e62a36798404/report/index.html

@AshinGau

AshinGau commented May 17, 2024

Copy link
Copy Markdown
Member

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label May 17, 2024
@github-actions

github-actionsBot commented May 17, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actionsBot commented May 17, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

morningman
morningman approved these changes May 19, 2024
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman
morningmanforce-pushed the paimon_deletion_vector branch from 46411e1 to f39a510CompareMay 19, 2024 14:39
@morningman

morningman commented May 19, 2024

Copy link
Copy Markdown
Contributor

run buildall

@github-actions

github-actionsBot commented May 19, 2024

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot

doris-robot commented May 19, 2024

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 35.69% (9008/25242)
Line Coverage: 27.35% (74502/272394)
Region Coverage: 26.59% (38531/144885)
Branch Coverage: 23.43% (19663/83920)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f39a510abda2fd1f788aeb0fa3ffe1a2e8ff627d_f39a510abda2fd1f788aeb0fa3ffe1a2e8ff627d/report/index.html

morningman
morningman approved these changes May 20, 2024
View reviewed changes

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@AshinGau
AshinGau merged commit 1a5933c into apache:masterMay 20, 2024
morningman pushed a commit to morningman/doris that referenced this pull request May 22, 2024
@suxiaogang223@morningman
[feature](Paimon) support deletion vector for Paimon naive reader (ap…
a023938
…ache#34743)
support deletion vector for Paimon naive reader
@morningmanmorningman mentioned this pull request May 22, 2024
Merged
morningman added a commit that referenced this pull request May 22, 2024
@morningman@suxiaogang223
[feature](Paimon) support deletion vector for Paimon naive reader (#3…
adc364a
…4743) (#35241)
bp #34743
Co-authored-by: 苏小刚 <suxiaogang223@icloud.com>
dataroaring pushed a commit that referenced this pull request May 26, 2024
@suxiaogang223@dataroaring
[feature](Paimon) support deletion vector for Paimon naive reader (#3…
2151cec
…4743)
support deletion vector for Paimon naive reader
@suxiaogang223
suxiaogang223 deleted the paimon_deletion_vector branch August 15, 2024 08:49
@suxiaogang223suxiaogang223 mentioned this pull request Mar 31, 2025
Merged
16 tasks
morningman pushed a commit that referenced this pull request Apr 2, 2025
@suxiaogang223
[fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe (#…
688e756
…49645)
### What problem does this PR solve?
Related PR: #34743 Problem Summary:
The problem occurs because FE passes the deletion file URL with 'oss://'
prefix to BE, which cannot recognize this URL format. The solution is to
convert the 'oss://' prefix to 's3://' prefix in FE's PaimonScanNode
using LocationPath class to ensure BE can properly read the file.
github-actionsBot pushed a commit that referenced this pull request Apr 2, 2025
@suxiaogang223Your Name
[fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe (#…
52a6bdd
…49645)
### What problem does this PR solve?
Related PR: #34743 Problem Summary:
The problem occurs because FE passes the deletion file URL with 'oss://'
prefix to BE, which cannot recognize this URL format. The solution is to
convert the 'oss://' prefix to 's3://' prefix in FE's PaimonScanNode
using LocationPath class to ensure BE can properly read the file.
github-actionsBot pushed a commit that referenced this pull request Apr 2, 2025
@suxiaogang223Your Name
[fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe (#…
fa2d732
…49645)
### What problem does this PR solve?
Related PR: #34743 Problem Summary:
The problem occurs because FE passes the deletion file URL with 'oss://'
prefix to BE, which cannot recognize this URL format. The solution is to
convert the 'oss://' prefix to 's3://' prefix in FE's PaimonScanNode
using LocationPath class to ensure BE can properly read the file.
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@suxiaogang223koarz
[fix](paimon) Covert Paimon DeletionFile Path to StoragePath in fe (a…
6b722d6
…pache#49645)
### What problem does this PR solve?
Related PR: apache#34743 Problem Summary:
The problem occurs because FE passes the deletion file URL with 'oss://'
prefix to BE, which cannot recognize this URL format. The solution is to
convert the 'oss://' prefix to 's3://' prefix in FE's PaimonScanNode
using LocationPath class to ensure BE can properly read the file.
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@suxiaogang223@HappenLee
[feature](Paimon) support deletion vector for Paimon naive reader (ap…
d0ff15a
…ache#34743)
support deletion vector for Paimon naive reader
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actionsgithub-actions[bot]github-actions[bot] left review comments
@morningmanmorningmanmorningman approved these changes
@AshinGauAshinGauAshinGau approved these changes

Assignees

No one assigned

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.4-mergeddev/3.0.0-mergedmeta-changereviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@suxiaogang223@doris-robot@AshinGau@morningman@gavinchou@yiguolei