Uh oh!
There was an error while loading. Please reload this page.
[fix](array index) Correct null bitmap writing for inverted index - #47846
Conversation
hello-stephen
commented
Feb 12, 2025
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
airborne12
commented
Feb 12, 2025
run buildall |
doris-robot
commented
Feb 12, 2025
TPC-H: Total hot run time: 31516 ms |
doris-robot
commented
Feb 12, 2025
TPC-DS: Total hot run time: 189594 ms |
doris-robot
commented
Feb 12, 2025
ClickBench: Total hot run time: 30.46 s |
doris-robot
commented
Feb 12, 2025
TeamCity be ut coverage result: |
airborne12
commented
Feb 13, 2025
run buildall |
doris-robot
commented
Feb 13, 2025
TPC-H: Total hot run time: 31709 ms |
doris-robot
commented
Feb 13, 2025
TPC-DS: Total hot run time: 190122 ms |
doris-robot
commented
Feb 13, 2025
ClickBench: Total hot run time: 30.45 s |
doris-robot
commented
Feb 13, 2025
TeamCity be ut coverage result: |
airborne12
commented
Feb 13, 2025
run buildall |
doris-robot
commented
Feb 13, 2025
TPC-H: Total hot run time: 31527 ms |
doris-robot
commented
Feb 13, 2025
TPC-DS: Total hot run time: 183450 ms |
doris-robot
commented
Feb 13, 2025
ClickBench: Total hot run time: 30.43 s |
airborne12
commented
Feb 14, 2025
run buildall |
1 similar comment
airborne12
commented
Feb 14, 2025
run buildall |
doris-robot
commented
Feb 14, 2025
TPC-H: Total hot run time: 31599 ms |
doris-robot
commented
Feb 14, 2025
TPC-DS: Total hot run time: 189449 ms |
doris-robot
commented
Feb 14, 2025
ClickBench: Total hot run time: 29.75 s |
2f3799b to
2681acaComparePR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
…7846) This pull request focuses on improving the handling of null values in the inverted index writer and simplifying the codebase by removing redundant null map checks. The most important changes include removing unnecessary null map handling in several methods and ensuring proper null bitmap updates. Improvements to null value handling and code simplification: * [`be/src/olap/rowset/segment_v2/column_writer.cpp`](diffhunk://#diff-db6023c6e1df0c3616055f02e769cc20fcef7ee083cb3755cec1b661bb7b42ffL952-L958): Removed redundant null map handling in `Status ArrayColumnWriter::append_nullable` method. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L328-R328): Simplified `add_array_nulls` method to always return `Status::OK()`. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426): Added null map check before accessing elements in the loop to prevent potential null pointer dereference. [[1]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426) [[2]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L525-R531) * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165R513): Updated `_null_bitmap` in the `add_null_document` method to ensure proper null bitmap updates. * [`be/src/olap/task/index_builder.cpp`](diffhunk://#diff-df38b3b177cd231676ce7a405526b3419c543e29171143ddec02960a84a930c6L645-R645): Removed redundant null map handling in `Status IndexBuilder::_add_nullable` method.
…ache#47846) This pull request focuses on improving the handling of null values in the inverted index writer and simplifying the codebase by removing redundant null map checks. The most important changes include removing unnecessary null map handling in several methods and ensuring proper null bitmap updates. Improvements to null value handling and code simplification: * [`be/src/olap/rowset/segment_v2/column_writer.cpp`](diffhunk://#diff-db6023c6e1df0c3616055f02e769cc20fcef7ee083cb3755cec1b661bb7b42ffL952-L958): Removed redundant null map handling in `Status ArrayColumnWriter::append_nullable` method. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L328-R328): Simplified `add_array_nulls` method to always return `Status::OK()`. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426): Added null map check before accessing elements in the loop to prevent potential null pointer dereference. [[1]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426) [[2]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L525-R531) * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165R513): Updated `_null_bitmap` in the `add_null_document` method to ensure proper null bitmap updates. * [`be/src/olap/task/index_builder.cpp`](diffhunk://#diff-df38b3b177cd231676ce7a405526b3419c543e29171143ddec02960a84a930c6L645-R645): Removed redundant null map handling in `Status IndexBuilder::_add_nullable` method.
…ache#47846) This pull request focuses on improving the handling of null values in the inverted index writer and simplifying the codebase by removing redundant null map checks. The most important changes include removing unnecessary null map handling in several methods and ensuring proper null bitmap updates. Improvements to null value handling and code simplification: * [`be/src/olap/rowset/segment_v2/column_writer.cpp`](diffhunk://#diff-db6023c6e1df0c3616055f02e769cc20fcef7ee083cb3755cec1b661bb7b42ffL952-L958): Removed redundant null map handling in `Status ArrayColumnWriter::append_nullable` method. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L328-R328): Simplified `add_array_nulls` method to always return `Status::OK()`. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426): Added null map check before accessing elements in the loop to prevent potential null pointer dereference. [[1]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426) [[2]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L525-R531) * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165R513): Updated `_null_bitmap` in the `add_null_document` method to ensure proper null bitmap updates. * [`be/src/olap/task/index_builder.cpp`](diffhunk://#diff-df38b3b177cd231676ce7a405526b3419c543e29171143ddec02960a84a930c6L645-R645): Removed redundant null map handling in `Status IndexBuilder::_add_nullable` method.
…ache#47846) This pull request focuses on improving the handling of null values in the inverted index writer and simplifying the codebase by removing redundant null map checks. The most important changes include removing unnecessary null map handling in several methods and ensuring proper null bitmap updates. Improvements to null value handling and code simplification: * [`be/src/olap/rowset/segment_v2/column_writer.cpp`](diffhunk://#diff-db6023c6e1df0c3616055f02e769cc20fcef7ee083cb3755cec1b661bb7b42ffL952-L958): Removed redundant null map handling in `Status ArrayColumnWriter::append_nullable` method. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L328-R328): Simplified `add_array_nulls` method to always return `Status::OK()`. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426): Added null map check before accessing elements in the loop to prevent potential null pointer dereference. [[1]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426) [[2]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L525-R531) * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165R513): Updated `_null_bitmap` in the `add_null_document` method to ensure proper null bitmap updates. * [`be/src/olap/task/index_builder.cpp`](diffhunk://#diff-df38b3b177cd231676ce7a405526b3419c543e29171143ddec02960a84a930c6L645-R645): Removed redundant null map handling in `Status IndexBuilder::_add_nullable` method.
… is null (apache#48231) ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#47846 Problem Summary: Fix wrong adding array index for building index.
…ache#47846) This pull request focuses on improving the handling of null values in the inverted index writer and simplifying the codebase by removing redundant null map checks. The most important changes include removing unnecessary null map handling in several methods and ensuring proper null bitmap updates. Improvements to null value handling and code simplification: * [`be/src/olap/rowset/segment_v2/column_writer.cpp`](diffhunk://#diff-db6023c6e1df0c3616055f02e769cc20fcef7ee083cb3755cec1b661bb7b42ffL952-L958): Removed redundant null map handling in `Status ArrayColumnWriter::append_nullable` method. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L328-R328): Simplified `add_array_nulls` method to always return `Status::OK()`. * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426): Added null map check before accessing elements in the loop to prevent potential null pointer dereference. [[1]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L429-R426) [[2]](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165L525-R531) * [`be/src/olap/rowset/segment_v2/inverted_index_writer.cpp`](diffhunk://#diff-97781916b276f771710ab520c79ca29d5e4e331296fad7573fc9933a376dc165R513): Updated `_null_bitmap` in the `add_null_document` method to ensure proper null bitmap updates. * [`be/src/olap/task/index_builder.cpp`](diffhunk://#diff-df38b3b177cd231676ce7a405526b3419c543e29171143ddec02960a84a930c6L645-R645): Removed redundant null map handling in `Status IndexBuilder::_add_nullable` method.
… is null (apache#48231) ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#47846 Problem Summary: Fix wrong adding array index for building index.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #38907
Problem Summary:
This pull request focuses on improving the handling of null values in the inverted index writer and simplifying the codebase by removing redundant null map checks. The most important changes include removing unnecessary null map handling in several methods and ensuring proper null bitmap updates.
Improvements to null value handling and code simplification:
be/src/olap/rowset/segment_v2/column_writer.cpp: Removed redundant null map handling inStatus ArrayColumnWriter::append_nullablemethod.be/src/olap/rowset/segment_v2/inverted_index_writer.cpp: Simplifiedadd_array_nullsmethod to always returnStatus::OK().be/src/olap/rowset/segment_v2/inverted_index_writer.cpp: Added null map check before accessing elements in the loop to prevent potential null pointer dereference. [1][2]be/src/olap/rowset/segment_v2/inverted_index_writer.cpp: Updated_null_bitmapin theadd_null_documentmethod to ensure proper null bitmap updates.be/src/olap/task/index_builder.cpp: Removed redundant null map handling inStatus IndexBuilder::_add_nullablemethod.Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)