GH-50567: [C++] Introduce JsonWriter and migrate integration JSON writer by Reranko05 · Pull Request #50568 · apache/arrow

Reranko05 · 2026-07-21T04:46:16Z

What changes are included?

This PR introduces a reusable JsonWriter wrapper around simdjson's JSON builder API and migrates the integration JSON writer to use it instead of RapidJSON.

Specifically, this PR:

Adds a reusable JsonWriter abstraction in arrow/json.
Migrates the integration JSON writer implementation to JsonWriter.
Updates the integration tests to use JsonWriter.
Adds unit tests for JsonWriter.

This is part of the incremental migration from RapidJSON to simdjson.

Performance

I compared the serialization performance of IntegrationJsonWriter::WriteRecordBatch() + Finish() using a temporary benchmark with a 1M-row RecordBatch. The benchmark was run 10 times on both main and this branch.

Branch	Average Time
`main`	2157.4 ms
This PR	2153.6 ms

No measurable performance regression was observed.

Are these changes tested?

Yes.

I added unit tests for JsonWriter and verified that both JSON and integration tests pass locally:

arrow-json-test
arrow-json-integration-test

Are there any user-facing changes?

No.

GitHub Issue: [C++] Introduce JsonWriter and migrate integration JSON writer #50567

github-actions · 2026-07-21T04:46:40Z

⚠️ GitHub issue #50567has been automatically assigned in GitHub to PR creator.

Reranko05 · 2026-07-21T06:29:27Z

@kou could you review this PR when you have a chance?

kou

Could you check performance difference?

Reranko05 · 2026-07-21T07:31:38Z

@kou, I compared the serialization performance of IntegrationJsonWriter::WriteRecordBatch() + Finish() using a temporary benchmark with a 1M-row RecordBatch. I ran the benchmark 10 times on both main and this branch. After excluding one obvious outlier from each run, the averages were:

main: 2157.4 ms
this branch: 2153.6 ms

I didn't observe any measurable performance regression.

Reranko05 · 2026-07-21T08:38:50Z

@kou I investigated the failing workflows and they appear to be unrelated to this PR. Could you please verify?

rok

Thanks for working on this @Reranko05! Exciting to see this work!

We should probably add meson support for json_writer, see (cpp/src/arrow/meson.build, cpp/src/arrow/json/meson.build, cpp/src/arrow/integration/meson.build).

rok · 2026-07-21T12:50:27Z

Another thought - do we have conbench cpp micro benchmarks to monitor improvement of json parsing and writing?

Copilot

Pull request overview

This PR adds a reusable arrow::json::JsonWriter wrapper around simdjson’s builder API and migrates the C++ integration JSON writer (and its tests) away from RapidJSON’s writer, as part of the incremental RapidJSON → simdjson migration.

Changes:

Added arrow::json::JsonWriter (header/impl) plus unit tests.
Migrated integration JSON serialization code paths to emit JSON using JsonWriter.
Updated CMake/test linkage and CI environment to ensure simdjson is available/linked where needed.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cpp/src/arrow/json/json_writer.h	Introduces the `JsonWriter` public API.
cpp/src/arrow/json/json_writer.cc	Implements `JsonWriter` on top of simdjson builder primitives.
cpp/src/arrow/json/json_writer_test.cc	Adds unit coverage for basic writer operations.
cpp/src/arrow/json/CMakeLists.txt	Adds the new unit test and links simdjson for the test target.
cpp/src/arrow/integration/json_internal.h	Updates writer function signatures to accept `arrow::json::JsonWriter*`.
cpp/src/arrow/integration/json_internal.cc	Migrates integration JSON emission to `JsonWriter` (writer-side).
cpp/src/arrow/integration/json_integration.cc	Switches `IntegrationJsonWriter` implementation to `JsonWriter`.
cpp/src/arrow/integration/json_integration_test.cc	Updates integration tests to use `JsonWriter` for schema/array JSON generation.
cpp/src/arrow/integration/CMakeLists.txt	Links simdjson where integration tests/executables now depend on it.
cpp/src/arrow/CMakeLists.txt	Adds `json/json_writer.cc` to Arrow JSON sources and links simdjson for integration targets.
ci/docker/ubuntu-24.04-cpp.dockerfile	Sets `simdjson_SOURCE=BUNDLED` to ensure simdjson availability in CI images.

+ ArrayWriter(const std::string& name, const Array& array, JsonWriter* writer)
 : name_(name), array_(array), writer_(writer) {}


Reranko05 · 2026-07-21T14:35:10Z

Another thought - do we have conbench cpp micro benchmarks to monitor improvement of json parsing and writing?

I wasn't able to find any existing Conbench C++ microbenchmarks covering JSON parsing or writing. I did run a local benchmark comparing IntegrationJsonWriter::WriteRecordBatch() + Finish() on this branch against main and didn't observe any measurable regression. If I missed an existing Conbench benchmark, please let me know and I will run this against it.

rok · 2026-07-21T15:09:34Z

If I missed an existing Conbench benchmark, please let me know and I will run this against it.

It then seems we currently don't benchmark serializing and deserializing json. It would be useful to introduce some, but perhaps that is out of scope for this ticket.

This reverts commit 3265cf1.

Copilot