Add GetPreallocatedOutput to KernelContext.

[Eric Crawford (ericcraw)]

Description

Add KernelContext::GetPreallocatedOutput and the corresponding C API,
KernelContext_GetPreallocatedOutput.

The API allows an execution provider to query whether the caller supplied a
preallocated output tensor for the current kernel output and retrieve it as a
borrowed OrtValue. It does not allocate, resize, or replace output tensors.
The C++ wrapper and runtime execution-frame tracking are included, along with
tests covering Run, IoBinding, missing user outputs, invalid indices, and
direct writes to the caller-provided buffer.

Motivation and Context

Execution providers need to efficiently handle dynamically shaped models where
the caller knows the expected output shape but the EP cannot quickly or easily
determine it before inference execution. This is particularly important for
transformer models with large KV caches.

The new API allows an EP to reuse a caller-provided output buffer directly,
avoiding an intermediate allocation and subsequent copy. Since the API returns
a borrowed buffer without performing dynamic output-shape validation, the EP
must validate the tensor's element type and shape before writing to it.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds kernel-context access to preallocated outputs, enabling EPs to validate and write directly into caller-provided buffers.

Changes:

• Adds C, C++, and internal runtime APIs.
• Integrates direct-buffer use into the example plugin EP.
• Adds Run, IoBinding, missing-output, and validation tests.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
onnxruntime/test/autoep/test_execution.cc	Adds API and preallocated-output tests.
onnxruntime/test/autoep/test_autoep_utils.h	Defines new test hooks.
onnxruntime/test/autoep/test_autoep_utils.cc	Loads the new hooks.
onnxruntime/test/autoep/library/example_plugin_ep/example_plugin_ep_library.lds	Exports hook symbols.
onnxruntime/test/autoep/library/example_plugin_ep/ep.cc	Writes directly to validated preallocated outputs.
onnxruntime/test/autoep/library/example_plugin_ep/ep_test_hooks.h	Declares output-query hooks.
onnxruntime/test/autoep/library/example_plugin_ep/ep_test_hooks.cc	Implements output-query tracking.
onnxruntime/core/session/ort_apis.h	Declares the C API implementation.
onnxruntime/core/session/onnxruntime_c_api.cc	Adds the API function pointer.
onnxruntime/core/session/custom_ops.cc	Implements C API validation and lookup.
onnxruntime/core/framework/op_kernel.cc	Retrieves allocated output values.
onnxruntime/core/framework/op_kernel_context_internal.h	Exposes internal retrieval.
include/onnxruntime/core/session/onnxruntime_cxx_inline.h	Implements the C++ wrapper.
include/onnxruntime/core/session/onnxruntime_cxx_api.h	Declares the C++ API.
include/onnxruntime/core/session/onnxruntime_c_api.h	Documents and declares the public C API.
include/onnxruntime/core/framework/op_kernel_context.h	Declares framework-level retrieval.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Chi Lo (chilo-ms)]

Looks good to me except one minor comment.

If an EP finds that the value returned by  KernelContext_GetPreallocatedOutput  has an incompatible shape and then calls KernelContext_GetOutput with the computed shape, ORT sees that the output slot is already allocated, detects the shape mismatch, and returns an error. It does not resize or replace the preallocated tensor. Type, rank, and static-dimension mismatches for top-level outputs are generally rejected during pre-run validation.

Could we document this explicitly? In particular, the example plugin’s “fall back to  GetOutput() on a mismatch” wording might imply that ORT will allocate a replacement buffer. It would be clearer to state that the EP must not write to an incompatible tensor and that calling  KernelContext_GetOutput may reject an incompatible preallocated output rather than replace it.