C API memory leak Asan

[KilLogic98]

Describe the issue

I'm a rookie with ONNX Runtime. – this issue is urgent for my development. Your prompt reply would be much appreciated.

When repeatedly creating and releasing an OrtEnv (calling CreateEnv and ReleaseEnv in a loop), the program does not free all allocated memory. AddressSanitizer (LeakSanitizer) reports a small but persistent leak of 144 bytes (3 allocations) per iteration or in total, depending on the test.

[root@localhost build]# ./minimal_leak_test
Completed 10 iterations without crash.

=================================================================
==231965==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 64 byte(s) in 1 object(s) allocated from:
#0 0xffffb8fe91f0 in __interceptor_calloc (/usr/lib64/libasan.so.6+0xa51f0)
#1 0xffffb8b56af8 (/usr/lib64/libonnxruntime.so.1.17.0+0x819af8)
#2 0xffffb8211dac (/usr/lib64/libc.so.6+0x83dac)
#3 0xffffb8450f8c (/usr/lib64/libonnxruntime.so.1.17.0+0x113f8c)
#4 0xffffb8451eb4 (/usr/lib64/libonnxruntime.so.1.17.0+0x114eb4)
#5 0xffffb83f4164 (/usr/lib64/libonnxruntime.so.1.17.0+0xb7164)
#6 0xffffb83c215c (/usr/lib64/libonnxruntime.so.1.17.0+0x8515c)
#7 0x400b98 in main /opt/SQL/leak/build/minimal_leak_test.c:14
#8 0xffffb81b9038 (/usr/lib64/libc.so.6+0x2b038)
#9 0xffffb81b9110 in __libc_start_main (/usr/lib64/libc.so.6+0x2b110)
#10 0x40096c in _start (/opt/SQL/leak/build/minimal_leak_test+0x40096c)

Direct leak of 64 byte(s) in 1 object(s) allocated from:
#0 0xffffb8fe91f0 in __interceptor_calloc (/usr/lib64/libasan.so.6+0xa51f0)
#1 0xffffb8b56ae4 (/usr/lib64/libonnxruntime.so.1.17.0+0x819ae4)
#2 0xffffb8211dac (/usr/lib64/libc.so.6+0x83dac)
#3 0xffffb8450f8c (/usr/lib64/libonnxruntime.so.1.17.0+0x113f8c)
#4 0xffffb8451eb4 (/usr/lib64/libonnxruntime.so.1.17.0+0x114eb4)
#5 0xffffb83f4164 (/usr/lib64/libonnxruntime.so.1.17.0+0xb7164)
#6 0xffffb83c215c (/usr/lib64/libonnxruntime.so.1.17.0+0x8515c)
#7 0x400b98 in main /opt/SQL/leak/build/minimal_leak_test.c:14
#8 0xffffb81b9038 (/usr/lib64/libc.so.6+0x2b038)
#9 0xffffb81b9110 in __libc_start_main (/usr/lib64/libc.so.6+0x2b110)
#10 0x40096c in _start (/opt/SQL/leak/build/minimal_leak_test+0x40096c)

Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0xffffb8fe91f0 in __interceptor_calloc (/usr/lib64/libasan.so.6+0xa51f0)
#1 0xffffb8b56ad0 (/usr/lib64/libonnxruntime.so.1.17.0+0x819ad0)
#2 0xffffb8211dac (/usr/lib64/libc.so.6+0x83dac)
#3 0xffffb8450f8c (/usr/lib64/libonnxruntime.so.1.17.0+0x113f8c)
#4 0xffffb8451eb4 (/usr/lib64/libonnxruntime.so.1.17.0+0x114eb4)
#5 0xffffb83f4164 (/usr/lib64/libonnxruntime.so.1.17.0+0xb7164)
#6 0xffffb83c215c (/usr/lib64/libonnxruntime.so.1.17.0+0x8515c)
#7 0x400b98 in main /opt/SQL/leak/build/minimal_leak_test.c:14
#8 0xffffb81b9038 (/usr/lib64/libc.so.6+0x2b038)
#9 0xffffb81b9110 in __libc_start_main (/usr/lib64/libc.so.6+0x2b110)
#10 0x40096c in _start (/opt/SQL/leak/build/minimal_leak_test+0x40096c)

SUMMARY: AddressSanitizer: 144 byte(s) leaked in 3 allocation(s).

To reproduce

// minimal_leak_test.c
#include <onnxruntime_c_api.h>
#include <stdio.h>
#include <stdlib.h>

int main() {
const OrtApiBase* api_base = OrtGetApiBase();
const OrtApi* ort = api_base->GetApi(ORT_API_VERSION);
const int iterations = 10;
for (int i = 0; i < iterations; ++i) {
OrtEnv* env = NULL;
OrtStatus* status = ort->CreateEnv(ORT_LOGGING_LEVEL_WARNING, "test", &env);
if (status) {
fprintf(stderr, "CreateEnv failed at iteration %d\n", i);
ort->ReleaseStatus(status);
return 1;
}
ort->ReleaseEnv(env);
}
printf("Completed %d iterations without crash.\n", iterations);
return 0;
}

gcc -g -fsanitize=address -I../include minimal_leak_test.c -L../lib -lonnxruntime -o minimal_leak_test ./minimal_leak_test

Urgency

urgent

Platform

Linux

OS Version

GNU

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.16.0-1.22.0

ONNX Runtime API

C

Architecture

ARM64

Execution Provider

Other / Unknown

Execution Provider Library Version

No response

[edgchen1]

I wasn't able to reproduce the leak on an x64 Linux machine. I don't have an aarch64 Linux machine readily available.

are you able to build ORT from source with the debug symbols?

running something like this
build.sh --config Debug --build_dir ./build --build_shared_lib --update --build --parallel --target onnxruntime
should produce libonnxruntime.so in the ./build/Debug directory.

maybe running with that will show a more informative stacktrace.

[tairenpiao]

I don't think this is actually a leak. The giveaway is in your own output: you ran 10 iterations but LSan still only reports 3 objects (one each), so it isn't growing per loop.

That 144 bytes is almost certainly the one-time init ORT does the first time CreateEnv is called (registering the op schemas, setting up the symbolizer, and so on). It's behind a std::call_once and kept alive for the whole process on purpose, so LSan reports it as still-reachable at exit. That'd also explain why Edward Chen (@edgchen1) couldn't reproduce it as a real leak on x64.

Easy way to confirm: bump the loop to 100. If it's still 144 bytes, that's it, not a leak.

[KilLogic98]

Edward Chen (@edgchen1) I also couldn't reproduce this issue on x64. As for building ORT from source with debug symbols, I am unable to do so, as my system is located in a network‑isolated environment without direct internet connectivity, making such build procedures difficult.

[KilLogic98]

Tairen Piao (@tairenpiao) Thanks for your explanation. After increasing the loop iterations to 1000, the reported leak remains at 144 bytes. No leak is observed on x64, yet it appears on aarch64 – suggesting this is not a genuine code defect. Can this be considered a conclusion?

[edgchen1]

it might be related to a cpuinfo leak that was fixed recently.

#28245

there's a nightly NuGet build with the change you can try. the .nupkg file can be unzipped. Linux aarch64 binaries are in runtimes/linux-arm64/native and headers are in build/native/include.

https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly/NuGet/Microsoft.ML.OnnxRuntime/overview/1.28.0-dev-20260625-0511-d4b01e4373

otherwise, it should be in the next release.

[tairenpiao]

Yeah, I think that's conclusive. 1000 iterations and it's still flat at 144 bytes, so nothing is actually leaking per call. It's just one-time init that sticks around for the life of the process.

edgchen1's pointer looks right too: #28245 just landed (updates cpuinfo and adds cpuinfo_deinitialize), and cpuinfo's setup is exactly the kind of aarch64-only one-shot allocation that'd show up here but not on x64. I'd grab that nightly and re-run with it. My bet is the 144 bytes goes away.

[microsoft-github-policy-service]

Applying stale label due to no activity in 30 days. Please react to the issue by assigning it or labeling it as 'contributions welcome', 'documentation', 'feature request', 'regression', or 'no stale'.

[microsoft-github-policy-service]

Applying stale label due to no activity in 30 days. Please react to the issue by assigning it or labeling it as 'contributions welcome', 'documentation', 'feature request', 'regression', or 'no stale'.

[microsoft-github-policy-service]

Closing issue due to no activity in 30 days