Skip to content

GH-50859: [C++][Parquet] Move JsonWriter to simdjson utilities - #50990

Merged
pitrou merged 7 commits into
apache:mainfrom
rok:gh-50859-parquet-without-json
Aug 31, 2026
Merged

GH-50859: [C++][Parquet] Move JsonWriter to simdjson utilities#50990
pitrou merged 7 commits into
apache:mainfrom
rok:gh-50859-parquet-without-json

Conversation

@rok

@rokrok commented Aug 25, 2026

Copy link
Copy Markdown
Member

Rationale for this change

Parquet uses JsonWriter when ARROW_JSON=OFF, but its implementation was only built with Arrow JSON, causing link failures.

What changes are included in this PR?

Move JsonWriter to the simdjson utilities and update its callers and CMake/Meson builds.

Are these changes tested?

Yes. CMake shared/static and Meson Parquet builds pass with JSON disabled. Unit tests and pre-commit checks also pass.

Are there any user-facing changes?

No. This only fixes the affected build configuration.

AI disclosure - this was AI generated to test alternative approach to #50900.

@pitrou

Copy link
Copy Markdown
Member

The CI failures need fixing.

@rok

rok commented Aug 25, 2026

Copy link
Copy Markdown
MemberAuthor

@pitrou done.

@Reranko05Reranko05 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rok Since simdjson_internal.cc now exists, shouldn't all the non-template functions currently in simdjson_internal.h be moved into simdjson_internal.cc, leaving only declarations in the header?

@rok

rok commented Aug 25, 2026

Copy link
Copy Markdown
MemberAuthor

@rok Since simdjson_internal.cc now exists, shouldn't all the non-template functions currently in simdjson_internal.h be moved into simdjson_internal.cc, leaving only declarations in the header?

Done

@Reranko05Reranko05 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rok

rok commented Aug 25, 2026

Copy link
Copy Markdown
MemberAuthor

Again, this PR is purely AI generated. I will mark as ready for review once I review myself.

@pitrou

Copy link
Copy Markdown
Member

@rok Are you willing to prioritize this?

@rok

rok commented Aug 27, 2026

Copy link
Copy Markdown
MemberAuthor

@pitrou I'll review this in about an hour and ping again

@tadeja

Copy link
Copy Markdown
Member

@github-actions crossbow submit example-cpp-tutorial

@github-actions

Copy link
Copy Markdown

Revision: 973b4d2

Submitted crossbow builds: ursacomputing/crossbow @ actions-5765e8701c

TaskStatus
example-cpp-tutorialGitHub Actions

Parquet uses JsonWriter independently of the Arrow JSON module. Move the writer into the simdjson utilities so it is available whenever simdjson is enabled, including ARROW_JSON=OFF builds. Preserve the writer files as renames and update CMake, Meson, callers, and tests.
@rok
rokforce-pushed the gh-50859-parquet-without-json branch from 973b4d2 to 5a1f424CompareAugust 27, 2026 14:51
Comment threadcpp/src/arrow/util/CMakeLists.txt Outdated
SOURCES
json_writer_internal_test.cc
EXTRA_LINK_LIBS
simdjson::simdjson)

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use arrow::simdjson here instead? Both should work, but arrow alieas would hide the vendored/system simdjson.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if(SIMDJSON_VENDORED)
add_library(arrow::simdjson ALIAS simdjson)
else()
add_library(arrow::simdjson ALIAS simdjson::simdjson)
endif()

It seems better to use arrow::simdjson based on the above statements? cc @kou

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, changed to arrow::simdjson.

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Aug 27, 2026
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 27, 2026
@rok

rok commented Aug 27, 2026

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit example-cpp-tutorial

@github-actions

Copy link
Copy Markdown

Revision: 86e85bd

Submitted crossbow builds: ursacomputing/crossbow @ actions-e4578321bd

TaskStatus
example-cpp-tutorialGitHub Actions

@rok
rok marked this pull request as ready for review August 27, 2026 15:50
CopilotAI review requested due to automatic review settings August 31, 2026 13:41
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 31, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

Suppressed comments (1)

cpp/src/parquet/encryption/key_material.h:79

  • This removes an exported overload from the installed parquet/encryption/key_material.h, so existing source and binaries that call KeyMaterial::Parse(const arrow::json::internal::ObjectParser*) will no longer compile/link. That contradicts the stated absence of user-facing changes. Please preserve the existing parser type/overload while making its implementation available when simdjson is enabled, or explicitly treat and document this as an API/ABI break.
 static KeyMaterial Parse(const std::string& key_material_string);

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 31, 2026
CopilotAI review requested due to automatic review settings August 31, 2026 14:25
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 31, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 31, 2026
CopilotAI review requested due to automatic review settings August 31, 2026 14:50
@rok
rokforce-pushed the gh-50859-parquet-without-json branch from 6a88bfb to d15007aCompareAugust 31, 2026 14:50
@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Aug 31, 2026
@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Aug 31, 2026

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

@rok

rok commented Aug 31, 2026

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit -g cpp

@github-actions

Copy link
Copy Markdown

Revision: d15007a

Submitted crossbow builds: ursacomputing/crossbow @ actions-350780c611

TaskStatus
example-cpp-minimal-build-staticGitHub Actions
example-cpp-minimal-build-static-system-dependencyGitHub Actions
example-cpp-tutorialGitHub Actions
test-build-cpp-fuzzGitHub Actions
test-conda-cppGitHub Actions
test-conda-cpp-valgrindGitHub Actions
test-debian-13-cpp-amd64GitHub Actions
test-debian-13-cpp-i386GitHub Actions
test-debian-experimental-cpp-gcc-15GitHub Actions
test-fedora-42-cppGitHub Actions
test-ubuntu-22.04-cppGitHub Actions
test-ubuntu-22.04-cpp-bundledGitHub Actions
test-ubuntu-22.04-cpp-emscriptenGitHub Actions
test-ubuntu-22.04-cpp-no-threadingGitHub Actions
test-ubuntu-24.04-cppGitHub Actions
test-ubuntu-24.04-cpp-bundled-offlineGitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundledGitHub Actions
test-ubuntu-24.04-cpp-gcc-14GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formatsGitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizerGitHub Actions

@pitroupitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @rok !

@pitrou

Copy link
Copy Markdown
Member

CI failures are unrelated to this PR.

@pitrou
pitrou merged commit 06e3b9a into apache:mainAug 31, 2026
132 of 140 checks passed
@pitroupitrou removed the awaiting changes Awaiting changes label Aug 31, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

@rok@pitrou@tadeja@wgtmac@HuaHuaY@Reranko05