[EXPORTER] Fix Elasticsearch log exporter aborting on invalid UTF-8 by om7057 · Pull Request #4501 · open-telemetry/opentelemetry-cpp

om7057 · 2026-08-30T07:57:09Z

Summary

A log record's body or attributes may carry bytes that are not valid UTF-8 (a truncated multibyte sequence, a payload read in another encoding, a message built from a file path, etc). ElasticSearchRecordable::WriteValue stores the value as given, without validating or transcoding it. Export() then called nlohmann::json::dump() with its default strict error handler, which throws type_error.316 on invalid UTF-8. Since Export() is noexcept, that throw becomes std::terminate(), aborting the whole process rather than just failing the one export.

Fix

Switch to error_handler_t::replace, which substitutes U+FFFD for the invalid bytes instead of throwing. The record (and the rest of the bulk batch) is still exported, with just the invalid bytes replaced.

Test plan

Added ElasticsearchLogsExporterTests.ExportingARecordWithInvalidUtf8DoesNotAbort, using a fake injectable HttpClient/Session/Request/Response (the exporter already supports client injection) so the test doesn't need a real Elasticsearch instance.
Verified the test actually catches the regression: reverting the fix locally reproduces the exact std::terminate/abort from the issue.
Full es_log_record_exporter_test suite passes (3 tests, 3 pre-existing disabled tests unaffected).

A log record's body or attributes may carry bytes that are not valid UTF-8 (e.g. a truncated multibyte sequence, a payload read in another encoding, or a string built from a file path). ElasticSearchRecordable::WriteValue stores the value as given, without validating or transcoding it. Export() then called nlohmann::json::dump() with its default strict error handler, which throws on invalid UTF-8; since Export() is noexcept, that throw became std::terminate() and aborted the whole process, not just the one export. Switch to error_handler_t::replace, which substitutes U+FFFD for the invalid bytes instead of throwing, so the record (and the rest of the batch) is still exported. Fixes open-telemetry#4439

codecov · 2026-08-30T08:38:04Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.47%. Comparing base (a0d2d75) to head (6ff715c).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4501      +/-   ##
==========================================
+ Coverage   83.25%   83.47%   +0.23%     
==========================================
  Files         521      521              
  Lines       20379    20380       +1     
==========================================
+ Hits        16964    17011      +47     
+ Misses       3415     3369      -46

Files with missing lines	Coverage Δ
...orters/elasticsearch/src/es_log_record_exporter.cc	`47.73% <100.00%> (+35.52%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ctor> The fake Response's ForEachHeader overrides use nostd::function_ref directly; it was only reaching the test via a transitive include. <memory> and <vector> are only used transitively too and iwyu wants them dropped.

owent

LGTM, thanks.

marcalff · 2026-09-01T14:56:30Z

+// Regression test: a log record whose body carries bytes that are not valid UTF-8 used to
+// abort the process. ElasticSearchRecordable::WriteValue stores the value as given, and
+// Export() previously called nlohmann::json::dump() with its default strict error handler,
+// which throws on invalid UTF-8; since Export() is noexcept, that throw became
+// std::terminate(). The exporter now tolerates it instead of crashing.


No changes needed, but a nit comment for future PRs:

Comments in the CODE should be about what the code does.

Comments about what the code did before a change belongs to the PR or issue description, not in the code itself.

That said, thanks for the tests.

marcalff

LGTM, thanks for the fix and tests.

…ccepts Main grew its own FakeResponse, FakeRequest, FakeSession and FakeHttpClient in open-telemetry#4071 and open-telemetry#4501, in an unnamed namespace, and this branch already had four of the same names in another unnamed namespace in the same file. Reopening an unnamed namespace names the same namespace, so the rebase merged both without a conflict into four redefinitions that do not compile. Merged rather than renamed. Main's set is used once and answers with one fixed response; this branch's runs a script the case supplies, which is the general case. So the session and the client take a script and default to answering the way main's did, and the existing call site is untouched. The default body had to change too, which is not cosmetic. It read `{"errors": false, "failed" : 0}`, and this pull request stops reading `failed` and starts requiring one `items` result per submitted record, so that body is now rejected: with it, ExportingARecordWithInvalidUtf8DoesNotAbort returns kFailure and fails. The default is now a body a server could send for a one record batch, and the comment that described the old substring search is rewritten. Verified in both configurations, since the guards cut different code: 26 tests, 22 passed and 4 skipped without async export, 20 passed and 6 skipped with it, no failures in either. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>

Main's open-telemetry#4071 and open-telemetry#4501 put FakeResponse, FakeRequest, FakeSession and FakeHttpClient in an unnamed namespace at the top of this file, and this branch already had four of those names in a second unnamed namespace lower down. Reopening an unnamed namespace names the same namespace, so the rebase merged both with no conflict at all and left four redefinitions. One set now. The session and the client take a script and default to answering the way main's did, so main's own call site needs no edit, and DeferredSession and DeferredHttpClient are untouched. The default body stays as main wrote it. This branch does not change how the exporter decides success, so what main's cases send still passes here. Verified in both configurations with maintainer mode on: 19 cases, all passing without async export, 3 passing and 16 skipping with it. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>

Main's open-telemetry#4071 and open-telemetry#4501 put FakeResponse, FakeRequest, FakeSession and FakeHttpClient in an unnamed namespace at the top of this file, and this branch already had four of those names in a second unnamed namespace lower down. Reopening an unnamed namespace names the same namespace, so the rebase merged both with no conflict at all and left four redefinitions. One set now. The session and the client take a script and default to answering the way main's did, so main's own call site needs no edit. The script carries the handler as a shared_ptr rather than a reference, because the cases here have to keep it and send a second event to it, and the client keeps its on_create_session and on_cancel_all hooks, both empty by default. The default body stays as main wrote it. This branch does not change how the exporter decides success, so what main's cases send still passes here. Verified in both configurations with maintainer mode on: 12 cases, 12 passing with async export and 3 passing with 9 skipping without it. Removing CompleteOnce's compare and exchange turns six of them red, so the rewritten fixtures still discriminate. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>

om7057 requested a review from a team as a code owner August 30, 2026 07:57

owent approved these changes Sep 1, 2026

View reviewed changes

marcalff reviewed Sep 1, 2026

View reviewed changes

marcalff approved these changes Sep 1, 2026

View reviewed changes

Merge branch 'main' into fix/elasticsearch-invalid-utf8-abort

6ff715c

marcalff merged commit 8165c2d into open-telemetry:main Sep 1, 2026
77 checks passed

malkia mentioned this pull request Sep 1, 2026

[EXPORTER] Fix Elasticsearch log exporter aborting on invalid UTF-8 (… malkia/opentelemetry-cpp#1010

Merged

3 tasks

thc1006 mentioned this pull request Sep 8, 2026

[EXPORTER] Fix Elasticsearch log exporter Shutdown ignoring its timeout #4523

Open

Navigation Menu

Uh oh!

[EXPORTER] Fix Elasticsearch log exporter aborting on invalid UTF-8 - #4501

[EXPORTER] Fix Elasticsearch log exporter aborting on invalid UTF-8#4501
marcalff merged 3 commits into
open-telemetry:mainfrom
om7057:fix/elasticsearch-invalid-utf8-abort

om7057 commented Aug 30, 2026

Uh oh!

codecov Bot commented Aug 30, 2026 •

edited

Loading

Uh oh!

owent left a comment

Uh oh!

marcalff Sep 1, 2026

Uh oh!

marcalff left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

om7057 commented Aug 30, 2026

Summary

Fix

Test plan

Uh oh!

codecov Bot commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

owent left a comment

Choose a reason for hiding this comment

Uh oh!

marcalff Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

marcalff left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Aug 30, 2026 •

edited

Loading