[Improvement](memcpy) use assume_aligned to hint aligned memcpy by BiteTheDDDDt · Pull Request #60695 · apache/doris

BiteTheDDDDt · 2026-02-12T02:03:56Z

What problem does this PR solve?

This pull request improves the performance and correctness of fixed-size memory copy operations in hash map key handling by adding alignment-aware logic. The main changes introduce runtime alignment checks and use compiler hints to optimize memory copying, which can help leverage SIMD instructions and avoid undefined behavior due to misaligned accesses.

Hash map key handling improvements:

Added runtime checks for memory alignment in MethodKeysFixed to determine whether to use aligned or unaligned memory copy operations, improving safety and performance. [1][2][3]
Refactored lambda functions in key handling logic to support alignment-aware memory copying, replacing previous hardcoded alignment assumptions. [1][2][3]

Memory copy utility enhancements:

Updated memcpy_fixed in memcpy_small.h to use std::assume_aligned for aligned copies, providing better optimization opportunities for the compiler and ensuring correctness.
Added #include <memory> to support std::assume_aligned.

General code safety:

Added an early return in insert_keys_into_columns if num_rows == 0 to prevent unnecessary processing.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

reset reset

hello-stephen · 2026-02-12T02:04:01Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

BiteTheDDDDt · 2026-02-12T02:04:16Z

run buildall

Copilot

Pull request overview

This pull request aims to optimize fixed-size memory copy operations in hash map key handling by adding alignment-aware logic using std::assume_aligned. The PR introduces runtime checks to determine whether to use aligned or unaligned memory copy operations, and updates the memcpy_fixed utility function to leverage compiler hints for better SIMD optimization.

Changes:

Modified memcpy_fixed to use std::assume_aligned for aligned copies, providing optimization hints to the compiler
Refactored hash map key packing and unpacking logic to include runtime alignment checks before calling aligned memcpy
Added early return optimization when num_rows is 0

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
be/src/vec/common/memcpy_small.h	Updated memcpy_fixed to use std::assume_aligned for aligned copies; added #include
be/src/vec/common/hash_table/hash_map_context.h	Added runtime alignment checks in pack_fixeds and insert_keys_into_columns; added early return for num_rows == 0

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-12T02:08:27Z

+ }
+ };
+ auto foo = [&]<typename Fixed>(Fixed zero) {
+ if (reinterpret_cast<uintptr_t>(input_keys.data() + pos) % sizeof(Fixed) == 0) {


The alignment check incorrectly performs pointer arithmetic. The expression input_keys.data() + pos adds pos to a Key* pointer, which moves pos * sizeof(Key) bytes, not pos bytes. However, on line 734, the code correctly uses (char*)(&input_keys[j]) + pos to add pos bytes. The alignment check should also cast to char* before adding pos. The check should be: reinterpret_cast<uintptr_t>((char*)(input_keys.data()) + pos) % sizeof(Fixed) == 0
Suggested change
if (reinterpret_cast<uintptr_t>(input_keys.data() +pos) % sizeof(Fixed) ==0) {
if (reinterpret_cast<uintptr_t>(reinterpret_cast<char*>(input_keys.data()) +pos) % sizeof(Fixed) ==0) {

Copilot · 2026-02-12T02:08:27Z

+ auto foo = [&]<typename Fixed>(Fixed zero) {
+ if (reinterpret_cast<uintptr_t>(result_data + offset) % sizeof(T) == 0) {
+ goo.template operator()<Fixed, true>(zero);
+ } else {
+ goo.template operator()<Fixed, false>(zero);
+ }


When aligned=true, memcpy_fixed uses std::assume_aligned on both source and destination pointers, requiring both to be aligned to alignof(Fixed). However, the alignment check only verifies the destination pointer (result_data + offset), not the source pointer (data). If data is not aligned to alignof(Fixed), this creates undefined behavior. The code should either check both pointers for alignment or only use unaligned memcpy when either pointer is misaligned.

Copilot · 2026-02-12T02:08:27Z

+ auto foo = [&]<typename Fixed>(Fixed zero) {
+ if (reinterpret_cast<uintptr_t>(input_keys.data() + pos) % sizeof(Fixed) == 0) {
+ goo.template operator()<Fixed, true>(zero);
+ } else {
+ goo.template operator()<Fixed, false>(zero);
 }


When aligned=true, memcpy_fixed uses std::assume_aligned on both source and destination pointers. However, the alignment check only verifies the source pointer ((char*)(input_keys.data()) + pos), not the destination pointer (data + j * sizeof(Fixed)). If the destination is not aligned to alignof(Fixed), this creates undefined behavior. The code should check both pointers for alignment or use unaligned memcpy when either pointer is misaligned.

hello-stephen · 2026-02-12T09:55:57Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (25/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.77% (25968/36180)
Line Coverage	54.42% (271788/499445)
Region Coverage	51.76% (225521/435723)
Branch Coverage	53.32% (97043/182015)

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

BiteTheDDDDt · 2026-02-21T10:52:38Z

run buildallrun buildall

BiteTheDDDDt · 2026-02-21T10:52:42Z

run buildall

HappenLee

LGTM

github-actions · 2026-02-21T12:51:50Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-02-21T12:51:53Z

PR approved by anyone and no changes requested.

hello-stephen · 2026-02-21T14:45:36Z

BE UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.56% (19545/37183)
Line Coverage	36.14% (182214/504126)
Region Coverage	32.52% (141459/435040)
Branch Coverage	33.47% (61289/183108)

hello-stephen · 2026-02-21T16:10:02Z

BE Regression && UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.26% (20866/36442)
Line Coverage	40.29% (202622/502887)
Region Coverage	37.16% (163289/439425)
Branch Coverage	37.73% (69361/183812)

BiteTheDDDDt · 2026-02-24T07:30:15Z

run buildall

mrhhsg

LGTM

hello-stephen · 2026-02-24T11:01:54Z

BE Regression && UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.16% (20854/36486)
Line Coverage	40.20% (202222/503087)
Region Coverage	37.05% (162959/439787)
Branch Coverage	37.58% (69137/183976)

hello-stephen · 2026-02-24T11:09:04Z

BE Regression && UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.18% (20865/36490)
Line Coverage	40.20% (202402/503550)
Region Coverage	37.04% (163059/440197)
Branch Coverage	37.57% (69195/184179)

hello-stephen · 2026-02-24T12:18:00Z

BE Regression && UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.14% (20850/36490)
Line Coverage	40.19% (202358/503550)
Region Coverage	36.81% (162021/440197)
Branch Coverage	37.51% (69084/184179)

hello-stephen · 2026-02-24T12:26:09Z

BE Regression && UT Coverage Report

Increment line coverage 48.00% (12/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	57.14% (20850/36490)
Line Coverage	40.19% (202358/503550)
Region Coverage	36.81% (162021/440197)
Branch Coverage	37.51% (69084/184179)

hello-stephen · 2026-02-24T12:43:42Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (25/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	63.34% (23111/36490)
Line Coverage	46.59% (234625/503550)
Region Coverage	43.70% (192383/440197)
Branch Coverage	44.85% (82606/184179)

hello-stephen · 2026-02-24T13:53:54Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (25/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	63.34% (23111/36490)
Line Coverage	46.59% (234625/503550)
Region Coverage	43.70% (192383/440197)
Branch Coverage	44.85% (82606/184179)

BiteTheDDDDt · 2026-02-25T04:05:51Z

run buildall

hello-stephen · 2026-02-25T07:42:46Z

BE UT Coverage Report

Increment line coverage 46.43% (13/28) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.53% (19575/37265)
Line Coverage	36.14% (182568/505194)
Region Coverage	32.51% (141805/436166)
Branch Coverage	33.44% (61419/183660)

BiteTheDDDDt · 2026-02-25T08:53:39Z

run buildall

hello-stephen · 2026-02-25T09:00:02Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (32/32) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.60% (26134/36498)
Line Coverage	54.33% (273652/503694)
Region Coverage	51.76% (227923/440334)
Branch Coverage	53.11% (97852/184246)

hello-stephen · 2026-02-25T12:42:03Z

BE UT Coverage Report

Increment line coverage 45.16% (14/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.56% (19603/37293)
Line Coverage	36.18% (182894/505527)
Region Coverage	32.50% (141881/436550)
Branch Coverage	33.45% (61485/183818)

hello-stephen · 2026-02-25T14:09:17Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (31/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.27% (26763/36526)
Line Coverage	56.54% (284964/504027)
Region Coverage	53.98% (237884/440718)
Branch Coverage	55.63% (102580/184404)

hello-stephen · 2026-02-25T14:32:29Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (31/31) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	73.27% (26763/36526)
Line Coverage	56.53% (284925/504027)
Region Coverage	53.96% (237813/440718)
Branch Coverage	55.61% (102551/184404)

HappenLee

LGTM

mrhhsg

LGTM

github-actions · 2026-02-26T07:26:08Z

PR approved by at least one committer and no changes requested.

…he#60695) This pull request improves the performance and correctness of fixed-size memory copy operations in hash map key handling by adding alignment-aware logic. The main changes introduce runtime alignment checks and use compiler hints to optimize memory copying, which can help leverage SIMD instructions and avoid undefined behavior due to misaligned accesses.

use assume_aligned to hint aligned memcpy
b15a1fd
reset reset

CopilotAI review requested due to automatic review settings February 12, 2026 02:03

Copilotstarted reviewing on behalf of BiteTheDDDDt February 12, 2026 02:04 View session

CopilotAI reviewed Feb 12, 2026
View reviewed changes

Update be/src/vec/common/hash_table/hash_map_context.h
7caa807
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

HappenLee previously approved these changes Feb 21, 2026
View reviewed changes

github-actionsBot added approved Indicates a PR has been approved by one committer. reviewed labels Feb 21, 2026

mrhhsg previously approved these changes Feb 24, 2026
View reviewed changes

update
3b853df

BiteTheDDDDt dismissed stale reviews from mrhhsg and HappenLee via 3b853dfFebruary 25, 2026 04:05

github-actionsBot removed the approved Indicates a PR has been approved by one committer. label Feb 25, 2026

update
6dc0507

HappenLee approved these changes Feb 26, 2026
View reviewed changes

mrhhsg approved these changes Feb 26, 2026
View reviewed changes

github-actionsBot added the approved Indicates a PR has been approved by one committer. label Feb 26, 2026

BiteTheDDDDt merged commit d7ae570 into apache:masterFeb 26, 2026
29 of 31 checks passed

yiguolei added the dev/4.0.x label Feb 27, 2026

github-actionsBot added the dev/4.0.x-conflict label Feb 27, 2026

yiguolei mentioned this pull request Mar 23, 2026
pick memcpy alignement and datetime related pr to 4.1 #61620
Merged
16 tasks

yiguolei added dev/4.1.0-merged and removed dev/4.0.x dev/4.0.x-conflict labels Mar 24, 2026

	if (reinterpret_cast<uintptr_t>(input_keys.data() +pos) % sizeof(Fixed) ==0) {
	if (reinterpret_cast<uintptr_t>(reinterpret_cast<char*>(input_keys.data()) +pos) % sizeof(Fixed) ==0) {

Uh oh!

Conversation

BiteTheDDDDt commented Feb 12, 2026

What problem does this PR solve?

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Feb 12, 2026

Uh oh!

BiteTheDDDDt commented Feb 12, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

CopilotAIFeb 12, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAIFeb 12, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAIFeb 12, 2026

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Feb 12, 2026

BE Regression && UT Coverage Report

Uh oh!

BiteTheDDDDt commented Feb 21, 2026

Uh oh!

BiteTheDDDDt commented Feb 21, 2026

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actionsBot commented Feb 21, 2026

Uh oh!

github-actionsBot commented Feb 21, 2026

Uh oh!

hello-stephen commented Feb 21, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Feb 21, 2026

BE Regression && UT Coverage Report

Uh oh!

BiteTheDDDDt commented Feb 24, 2026

Uh oh!

mrhhsg left a comment

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Feb 24, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 24, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 24, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 24, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 24, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 24, 2026

BE Regression && UT Coverage Report

Uh oh!

BiteTheDDDDt commented Feb 25, 2026

Uh oh!

hello-stephen commented Feb 25, 2026

BE UT Coverage Report

Uh oh!

BiteTheDDDDt commented Feb 25, 2026

Uh oh!

hello-stephen commented Feb 25, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 25, 2026