Skip to content

GH-34335: [C++][Parquet] Optimize Decoding DELTA_LENGTH_BYTE_ARRAY - #34336

Closed
mapleFU wants to merge 4 commits into
apache:mainfrom
mapleFU:parquet/optimize-delta-length-decoding
Closed

GH-34335: [C++][Parquet] Optimize Decoding DELTA_LENGTH_BYTE_ARRAY#34336
mapleFU wants to merge 4 commits into
apache:mainfrom
mapleFU:parquet/optimize-delta-length-decoding

Conversation

@mapleFU

@mapleFUmapleFU commented Feb 24, 2023

Copy link
Copy Markdown
Member

Rationale for this change

According to #34323 . DELTA_LENGTH_BYTE_ARRAY is much more slower. So do some optimizations.

What changes are included in this PR?

Some tiny changes

Are these changes tested?

No more tests included, these code are already tested

Are there any user-facing changes?

No

@github-actions

Copy link
Copy Markdown

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@mapleFUmapleFU changed the title [C++][Parquet]: Optimize Decoding DELTA_LENGTH_BYTE_ARRAYGH-34335: [C++][Parquet]: Optimize Decoding DELTA_LENGTH_BYTE_ARRAYFeb 24, 2023
@mapleFUmapleFU changed the title GH-34335: [C++][Parquet]: Optimize Decoding DELTA_LENGTH_BYTE_ARRAYGH-34335: [C++][Parquet] Optimize Decoding DELTA_LENGTH_BYTE_ARRAYFeb 24, 2023
@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #34335has been automatically assigned in GitHub to PR creator.

@mapleFU
mapleFU marked this pull request as draft February 24, 2023 15:50
@mapleFU

Copy link
Copy Markdown
MemberAuthor

This patch is currently for poc and fast verification. If it works and #34323 is merged, I'll making code for it better :)

@mapleFU

Copy link
Copy Markdown
MemberAuthor
for (int i = 0; i < max_values; ++i) {
buffer[i].ptr = data_ptr;
data_ptr += buffer[i].len;
}

By the way, I don't know would this code cause cache problem or prevent from optimization, causing pipeline hazard or compiler do not optimize here. I'd do some survey.

@github-actionsgithub-actionsBot added the awaiting review Awaiting review label Apr 6, 2023
@mapleFUmapleFU closed this Apr 6, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++][Parquet] Improve the performance for Decoding DELTA_LENGTH_BYTE_ARRAY

1 participant

@mapleFU