feat(clients): add GenericOpenAIAgent for any OpenAI Chat Completions API endpoint by srmanda-cs · Pull Request #145 · microsoft/AIOpsLab

Ash Manda (srmanda-cs) · 2026-02-24T21:38:41Z

Summary

Closes#144

Adds a GenericOpenAIClient and GenericOpenAIAgent that work with any provider
implementing the OpenAI Chat Completions API (/v1/chat/completions), with both base_url and
model configurable at runtime. My motivating use case is the
Poe API,
but this should work with any OpenAI Chat Completions-compatible endpoint.

Changes

File	Change
`clients/utils/llm.py`	Added `GenericOpenAIClient` — uses `client.chat.completions.create`, accepts `base_url`, `model`, and `api_key`
`clients/generic_openai.py`	New `GenericOpenAIAgent` following the same pattern as existing agents
`clients/registry.py`	Registered `GenericOpenAIAgent` as `"generic"` in `AgentRegistry`
`clients/README.md`	Added client to the list and documented the three env vars
`.env.example`	Added `OPENAI_COMPATIBLE_API_KEY`, `OPENAI_COMPATIBLE_BASE_URL`, `OPENAI_COMPATIBLE_MODEL`

Breaking Changes

None. All changes are purely additive — no existing classes, agents, or configurations
were modified.

Notes for Reviewers

This is a draft PR and I'm very open to feedback! Specifically, I'd appreciate input on:

Naming: GenericOpenAIAgent / GenericOpenAIClient vs. something else (e.g. OpenAICompatible*)
Whether "generic" is a good registry key, or if there's a preferred convention
Whether the env var prefix OPENAI_COMPATIBLE_ is appropriate
Any patterns from the existing clients I should be mirroring more closely

Happy to revise anything before this is considered ready for review.

…ndpoint

…v.example

Jackson Clark (HacksonClark) · 2026-02-24T21:39:58Z

For testing, could you run the agent on the AIOpsLab benchmark and attach some logs here? Should be good to merge after that :)

Ash Manda (srmanda-cs) · 2026-02-24T21:43:25Z

@microsoft-github-policy-service agree

Ash Manda (srmanda-cs) · 2026-02-24T21:44:08Z

Will do ASAP!

…broad compatibility

…0 for thinking model compatibility

…tions API links - Update Poe doc URL from /responses-api to /openai-compatible-api in: - clients/generic_openai.py (module docstring) - clients/README.md (two occurrences) - .env.example (section heading and comment)

Ash Manda (srmanda-cs) · 2026-02-24T23:00:02Z

generic_openai_run.log

Here is the attached log file. It isn't a full run. Just a partial run, enough to say that the agent is working.

Jackson Clark (HacksonClark) · 2026-02-24T23:54:39Z

Logs look great!

Copilot

Pull request overview

Adds a new “generic” OpenAI-compatible client/agent so AIOpsLab can talk to any provider exposing an OpenAI-style Chat Completions endpoint, with base_url and model configurable via env vars or constructor args.

Changes:

Added GenericOpenAIClient to clients/utils/llm.py with configurable base_url, model, and api_key.
Introduced GenericOpenAIAgent (clients/generic_openai.py) and registered it under "generic" in AgentRegistry.
Documented configuration in clients/README.md and added sample env vars to .env.example.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`clients/utils/llm.py`	Adds `GenericOpenAIClient` using `OpenAI(..., base_url=...)` and Chat Completions calls.
`clients/generic_openai.py`	New agent wiring `GenericOpenAIClient` into the orchestrator flow.
`clients/registry.py`	Registers the new agent under the `"generic"` key.
`clients/README.md`	Documents the new generic agent and required env vars.
`.env.example`	Adds `OPENAI_COMPATIBLE_*` example configuration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-25T00:11:52Z

+ max_tokens: int = 16000,
+ ):
+ self.cache = Cache()
+ self.model = model or os.getenv("OPENAI_COMPATIBLE_MODEL", "gpt-4o")
+ self.max_tokens = max_tokens


max_tokens defaults to 16000 here, while the other Chat Completions clients in this module default to 1024. A high default is likely to exceed many providers’ limits and cause requests to fail unless callers override it; consider using a more conservative default (or reading an env var) to improve out-of-the-box compatibility.
Suggested change
max_tokens: int=16000,
):
self.cache=Cache()
self.model=modeloros.getenv("OPENAI_COMPATIBLE_MODEL", "gpt-4o")
self.max_tokens=max_tokens
max_tokens: Optional[int] =None,
):
self.cache=Cache()
self.model=modeloros.getenv("OPENAI_COMPATIBLE_MODEL", "gpt-4o")
ifmax_tokensisnotNone:
self.max_tokens=max_tokens
else:
env_max_tokens=os.getenv("OPENAI_COMPATIBLE_MAX_TOKENS")
self.max_tokens=int(env_max_tokens) ifenv_max_tokensisnotNoneelse1024

If the max tokens isn't set that high, a lot of api endpoints (like Poe) that support thinking will not be able to initialize, as the minimum token requirement is 4096. And just because max token requirement is high does not imply that the full token context will be used. It is a good idea to take in an environment variable, but then we would have to update the implementation for every client. I think, there is enough flexibility offered here.

Copilot · 2026-02-25T00:11:52Z

+ except Exception as e:
+ print(f"Exception: {repr(e)}")
+ raise e


In the exception handler, raise e resets the traceback context in Python. Use a bare raise to preserve the original stack trace (and consider using the project’s logging approach instead of print for errors).

This is standard implementation for every client, for consistency used the same block of code

Copilot · 2026-02-25T00:11:52Z

+"""Generic OpenAI-compatible chat client (with shell access) for AIOpsLab.
+
+This agent works with any provider that implements the OpenAI Chat Completions
+API endpoint (/v1/chat/completions), such as Poe
+(https://creator.poe.com/docs/external-applications/openai-compatible-api),
+standard OpenAI deployments, vLLM, LocalAI, or other compatible services.


The PR title/description references the OpenAI “Responses API”, but this agent (and GenericOpenAIClient) is implemented against the Chat Completions endpoint (/v1/chat/completions). Please align the PR wording/docs with what’s actually supported (either update the PR description/title to Chat Completions, or implement the Responses API).

Copilot · 2026-02-25T00:11:52Z

+ def init_context(self, problem_desc: str, instructions: str, apis: str):
+ """Initialize the context for the agent."""
+
+ self.shell_api = self._filter_dict(apis, lambda k, _: "exec_shell" in k)
+ self.submit_api = self._filter_dict(apis, lambda k, _: "submit" in k)
+ stringify_apis = lambda apis: "\n\n".join(


init_context types apis as str, but the implementation treats it as a mapping (calls .items() and filters by key). Updating the annotation to a Mapping[str, Any] (or similar) would make the contract clearer and avoid misleading type hints.

Standard implementation for every agent. Reused similar code block in generic OpenAI agent as well.

Yeah, this is a nit but making the change I feel.

I agree the type hint should be updated, but since this apis: str pattern is currently standard across multiple agent files, changing it just here would create inconsistency. To keep this PR focused on its original scope, how about I open a separate issue and a dedicated PR to update this type hint universally across all agents?

Makes sense

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Ash Manda (srmanda-cs) added 5 commits February 24, 2026 16:25

feat(clients): add GenericOpenAIClient for any OpenAI Responses API e…
a712ecf
…ndpoint

feat(clients): add GenericOpenAIAgent using GenericOpenAIClient
aa16259

feat(clients): register GenericOpenAIAgent in AgentRegistry
3f65e90

docs(clients): document GenericOpenAIAgent and its configuration
3ecefb8

chore(env): add OPENAI_COMPATIBLE_API_KEY, BASE_URL, and MODEL to .en…
27ecf69
…v.example

Ash Manda (srmanda-cs) added 3 commits February 24, 2026 17:27

fix(clients): switch GenericOpenAIClient to chat.completions API for …
c8696c2
…broad compatibility

fix(clients): increase GenericOpenAIClient default max_tokens to 1600…
3b1cc96
…0 for thinking model compatibility

Ash Manda (srmanda-cs) marked this pull request as ready for review February 25, 2026 00:03

CopilotAI review requested due to automatic review settings February 25, 2026 00:03

Copilotstarted reviewing on behalf of Ash Manda (srmanda-cs)February 25, 2026 00:04 View session

CopilotAI reviewed Feb 25, 2026
View reviewed changes

Ash Manda (srmanda-cs) changed the title ~~feat(clients): add GenericOpenAIAgent for any OpenAI Responses API endpoint~~feat(clients): add GenericOpenAIAgent for any OpenAI Chat Completions API endpointFeb 25, 2026

Update clients/utils/llm.py
7eb09d0
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Gagan Somashekar (gaganso) merged commit daa7961 into microsoft:mainFeb 25, 2026
1 check passed

Ash Manda (srmanda-cs) mentioned this pull request Feb 25, 2026
[Chore] Cleaning up and consolidating client code #146
Open

Ash Manda (srmanda-cs) deleted the feature/generic-openai-client branch February 25, 2026 22:03

-max_tokens: int=16000,
- ):
-self.cache=Cache()
-self.model=modeloros.getenv("OPENAI_COMPATIBLE_MODEL", "gpt-4o")
-self.max_tokens=max_tokens
+max_tokens: Optional[int] =None,
+ ):
+self.cache=Cache()
+self.model=modeloros.getenv("OPENAI_COMPATIBLE_MODEL", "gpt-4o")
+ifmax_tokensisnotNone:
+self.max_tokens=max_tokens
+else:
+env_max_tokens=os.getenv("OPENAI_COMPATIBLE_MAX_TOKENS")
+self.max_tokens=int(env_max_tokens) ifenv_max_tokensisnotNoneelse1024

Uh oh!

Conversation

Ash Manda (srmanda-cs) commented Feb 24, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Breaking Changes

Notes for Reviewers

Uh oh!

Jackson Clark (HacksonClark) commented Feb 24, 2026

Uh oh!

Ash Manda (srmanda-cs) commented Feb 24, 2026

Uh oh!

Ash Manda (srmanda-cs) commented Feb 24, 2026

Uh oh!

Ash Manda (srmanda-cs) commented Feb 24, 2026

Uh oh!

Jackson Clark (HacksonClark) commented Feb 24, 2026

Uh oh!

CopilotAI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

CopilotAIFeb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Ash Manda (srmanda-cs)Feb 25, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CopilotAIFeb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Ash Manda (srmanda-cs)Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAIFeb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Ash Manda (srmanda-cs)Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

CopilotAIFeb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Ash Manda (srmanda-cs)Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Gagan Somashekar (gaganso)Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Ash Manda (srmanda-cs)Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Gagan Somashekar (gaganso)Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Ash Manda (srmanda-cs) commented Feb 24, 2026•
edited
Loading

Ash Manda (srmanda-cs)Feb 25, 2026•
edited
Loading