Skip to content

GH-51029: [C++] Fix MapArray validation with unknown null count - #51093

Merged
pitrou merged 3 commits into
apache:mainfrom
AnuragRaut08:fix/maparray-cast-validity
Sep 2, 2026
Merged

GH-51029: [C++] Fix MapArray validation with unknown null count#51093
pitrou merged 3 commits into
apache:mainfrom
AnuragRaut08:fix/maparray-cast-validity

Conversation

@AnuragRaut08

@AnuragRaut08AnuragRaut08 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

MapArray validation uses MayHaveNulls() to check that the map child and keys contain no nulls. When an all-valid validity bitmap is present with an unknown null count, MayHaveNulls() can report that the array may contain nulls even though all values are valid.

During a cast of a MapArray, this can cause validation to return an invalid status which is passed to ARROW_CHECK_OK, resulting in a process abort instead of a recoverable error.

What changes are included in this PR?

  • Use GetNullCount() instead of MayHaveNulls() when validating the MapArray child and keys.
  • Add a regression test covering an all-valid validity bitmap with kUnknownNullCount.

Are these changes tested?

Yes, with existing and additional unit tests.

Are there any user-facing changes?

Only a bugfix.

This PR contains a "Critical Fix".

This fixes a bug that can cause a process crash when casting a valid MapArray whose keys carry an all-valid validity bitmap with an unknown null count. This is easily reproduced in Python:

importpyarrowaspam=pa.array([{"a": "1"}, {"b": "2"}], type=pa.map_(pa.string(), pa.string()))
m.filter(pa.array([False, True])).cast(pa.map_(pa.large_string(), pa.large_string()))

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #51029has been automatically assigned in GitHub to PR creator.

@pitroupitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this @AnuragRaut08 ! See comments below.

Also, there are other uses of MayHaveNulls in array_nested.cc, shouldn't we fix them as well?

Comment threadcpp/src/arrow/array/array_nested.cc
Comment thread.gitignore Outdated
Comment threadcpp/src/arrow/compute/kernels/scalar_cast_test.cc Outdated
@github-actionsgithub-actionsBot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Aug 31, 2026

@Reranko05Reranko05 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fix the lint by using:-
python -m pre_commit run clang-format --files <file_path>

@AnuragRaut08
AnuragRaut08force-pushed the fix/maparray-cast-validity branch from 0e4c2a2 to 4574e70CompareSeptember 1, 2026 15:49
@AnuragRaut08

Copy link
Copy Markdown
ContributorAuthor

Done, ran clang-format on the affected C++ files with pre-commit.

@AnuragRaut08

Copy link
Copy Markdown
ContributorAuthor

@pitrou I checked the other MayHaveNulls() uses in array_nested.cc. They are used for validating nullable list/map offsets when constructing arrays, rather than validating MapArray keys, so I left them unchanged.

@pitrou

Copy link
Copy Markdown
Member

@pitrou I checked the other MayHaveNulls() uses in array_nested.cc. They are used for validating nullable list/map offsets when constructing arrays, rather than validating MapArray keys, so I left them unchanged.

But the same issue applies: MayHaveNulls does not tell whether there are really nulls. So we should strengthen those checks as well.

@AnuragRaut08
AnuragRaut08force-pushed the fix/maparray-cast-validity branch from 4574e70 to aee42e9CompareSeptember 1, 2026 16:17
@AnuragRaut08

Copy link
Copy Markdown
ContributorAuthor

Thanks, addressed. I have updated the remaining MayHaveNulls() checks in array_nested.cc to use GetNullCount() != 0 and added regression coverage for an all-valid offsets bitmap with kUnknownNullCount. The TestMapArray.* tests pass.

@pitroupitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, LGTM.

Comment thread.gitignore Outdated
MANIFEST
compile_commands.json
build.ninja
build/

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
build/

@pitroupitrou added Critical Fix Bugfixes for security vulnerabilities, crashes, or invalid data. backport-candidate labels Sep 2, 2026
@pitrou

Copy link
Copy Markdown
Member

@AlenkaF FYI

@pitrou
pitrou merged commit a4e3909 into apache:mainSep 2, 2026
89 of 109 checks passed
@pitroupitrou removed the awaiting committer review Awaiting committer review label Sep 2, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-candidateComponent: C++Critical FixBugfixes for security vulnerabilities, crashes, or invalid data.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@AnuragRaut08@pitrou@Reranko05