Skip to content

[fix](hive)fix hive catalog miss partition that have special characters. - #42906

Merged
morningman merged 4 commits into
apache:masterfrom
hubgeter:hive_miss_partition
Nov 1, 2024
Merged

[fix](hive)fix hive catalog miss partition that have special characters.#42906
morningman merged 4 commits into
apache:masterfrom
hubgeter:hive_miss_partition

Conversation

@hubgeter

@hubgeterhubgeter commented Oct 30, 2024

Copy link
Copy Markdown
Contributor

Proposed changes

Previously, when processing partition values, Hive catalog parsed them in the URL format. However, this is different from the encoding method of Hive, which results in missing some partitions with special characters when reading the partition table.
This PR is to fix this problem.
Ref: common/src/java/org/apache/hadoop/hive/common/FileUtils.java:makePartName(List<String> partCols, List<String> vals,String defaultStr)

@doris-robot

Copy link
Copy Markdown

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@hubgeter

Copy link
Copy Markdown
ContributorAuthor

run buildall

@hubgeter

Copy link
Copy Markdown
ContributorAuthor

run buildall

@hubgeter
hubgeter marked this pull request as ready for review October 30, 2024 11:22
@morningman

Copy link
Copy Markdown
Contributor

run buildall

@hubgeter

Copy link
Copy Markdown
ContributorAuthor

run buildall

@hubgeter

Copy link
Copy Markdown
ContributorAuthor

run buildall

@morningmanmorningman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actionsgithub-actionsBot added the approved Indicates a PR has been approved by one committer. label Oct 31, 2024
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@morningman
morningman merged commit 03c625f into apache:masterNov 1, 2024
github-actionsBot pushed a commit that referenced this pull request Nov 1, 2024
…rs. (#42906)
## Proposed changes
Previously, when processing partition values, Hive catalog parsed them
in the URL format. However, this is different from the encoding method
of Hive, which results in missing some partitions with special
characters when reading the partition table.
This PR is to fix this problem.
Ref:
common/src/java/org/apache/hadoop/hive/common/FileUtils.java:`makePartName(List<String>
partCols, List<String> vals,String defaultStr)`
morningman pushed a commit to morningman/doris that referenced this pull request Nov 6, 2024
…rs. (apache#42906)
## Proposed changes
Previously, when processing partition values, Hive catalog parsed them
in the URL format. However, this is different from the encoding method
of Hive, which results in missing some partitions with special
characters when reading the partition table.
This PR is to fix this problem.
Ref:
common/src/java/org/apache/hadoop/hive/common/FileUtils.java:`makePartName(List<String>
partCols, List<String> vals,String defaultStr)`
morningman added a commit that referenced this pull request Nov 6, 2024
…rs. #42906 (#43318)
cherry pick from #42906
Co-authored-by: daidai <2017501503@qq.com>
dataroaring pushed a commit that referenced this pull request Nov 7, 2024
…ial characters. (#43070)
PR Body: ## Proposed changes
Previously, when processing partition values, Hive catalog parsed them
in the URL format. However, this is different from the encoding method
of Hive, which results in missing some partitions with special
characters when reading the partition table.
This PR is to fix this problem.
Ref:
common/src/java/org/apache/hadoop/hive/common/FileUtils.java:`makePartName(List<String>
partCols, List<String> vals,String defaultStr)`
Cherry-picked from #42906
Co-authored-by: daidai <2017501503@qq.com>
Co-authored-by: Calvin Kirs <kirs@apache.org>
morningman pushed a commit to morningman/doris that referenced this pull request Nov 18, 2024
…rs. (apache#42906)
## Proposed changes
Previously, when processing partition values, Hive catalog parsed them
in the URL format. However, this is different from the encoding method
of Hive, which results in missing some partitions with special
characters when reading the partition table.
This PR is to fix this problem.
Ref:
common/src/java/org/apache/hadoop/hive/common/FileUtils.java:`makePartName(List<String>
partCols, List<String> vals,String defaultStr)`
morningman added a commit that referenced this pull request Nov 21, 2024
…rs. #42906 (#44152)
cherry pick from #42906
---------
Co-authored-by: daidai <2017501503@qq.com>
@gavinchougavinchou mentioned this pull request Nov 26, 2024
@yiguoleiyiguolei mentioned this pull request Jan 19, 2025
hubgeter added a commit to hubgeter/doris that referenced this pull request Feb 13, 2025
@gavinchougavinchou mentioned this pull request Feb 18, 2025
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
…rs. (apache#42906)
## Proposed changes
Previously, when processing partition values, Hive catalog parsed them
in the URL format. However, this is different from the encoding method
of Hive, which results in missing some partitions with special
characters when reading the partition table.
This PR is to fix this problem.
Ref:
common/src/java/org/apache/hadoop/hive/common/FileUtils.java:`makePartName(List<String>
partCols, List<String> vals,String defaultStr)`
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approvedIndicates a PR has been approved by one committer.dev/2.1.8-mergeddev/3.0.3-mergedreviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@hubgeter@doris-robot@morningman@suxiaogang223@morrySnow