Skip to content

Local control for user_device environments - #161

Merged
abonneth merged 35 commits into
mainfrom
antoine/local-control
Jul 8, 2026
Merged

Local control for user_device environments#161
abonneth merged 35 commits into
mainfrom
antoine/local-control

Conversation

@abonneth

@abonnethabonneth commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Note

Medium Risk
Breaking environment client APIs and new on-machine command execution (browser/desktop) increase integration and operational risk; session cancellation on bridge loss can surprise callers.

Overview
Adds local execution for user_device environments and expands the environment model beyond cloud-only browsers.

The OpenAPI/SDK now treat environments as a web | desktop discriminated union. Browsers gain host (cloud / user_device), optional session_id to attach to an existing runner, and persist_browser_profile; vault/profile bindings are documented as cloud-only. Create/update environment calls take typed request bodies (CreateEnvironmentRequest_* / UpdateEnvironmentRequestBody_*) instead of flat Browser kwargs—a breaking SDK shape change.

New hai_agents_local package polls the platform command channel and dispatches to hai-drivers via SeleniumBrowserBridge (Chrome remote debugging) and PyautoguiDesktopBridge. Client/AsyncClient.sessions use LocalSessionsClient, which can auto-start bridges for inline agents (stamping session_id into unclaimed user_device envs), cancel the session if a bridge dies, and respect HAI_AUTO_BRIDGE. The CLI adds hai local browser|desktop for manual bridging.

Release 1.0.6 ships optional [browser] / [desktop] extras on hai-drivers, bundles hai_agents_local in the wheel, and updates discriminator default tests for the new environment kinds.

Reviewed by Cursor Bugbot for commit c854c5e. Bugbot is set up for automated code reviews on this repo. Configure here.

@abonneth
abonneth marked this pull request as ready for review June 29, 2026 18:15
@abonneth
abonneth requested a review from adeprezh as a code ownerJune 29, 2026 18:15
@hcompaihcompai deleted a comment from cm2435Jul 3, 2026
Comment threadsrc/hai_agents/local/runtime.py Outdated
Comment threadpyproject.toml
Comment threadsrc/hai_agents/local/runtime.py Outdated
@abonneth
abonnethforce-pushed the antoine/local-control branch 2 times, most recently from 14ca56a to bf57eb3CompareJuly 3, 2026 14:11
@abonnethabonneth changed the title Local control: in-SDK sidecar + desktop/browser driversLocal control: in-SDK sidecar for user_device environmentsJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
Comment threadsrc/hai_agents_local/browser.py
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
@hcompaihcompai deleted a comment from cursorBotJul 3, 2026
Comment threadsrc/hai_agents_local/sessions.py
…session after a bridge crash
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment threadsrc/hai_agents_local/bridge.py
Comment threadsrc/hai_agents_local/manager.py
Comment threadsrc/hai_agents_local/sessions.py
abonnethand others added 2 commits July 8, 2026 13:19
… kind
Co-authored-by: Cursor <cursoragent@cursor.com>
…c tokens off a running loop
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment threadsrc/hai_agents_local/sessions.py Outdated
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment threadsrc/hai_agents_local/sessions.py
…on still cancels
Co-authored-by: Cursor <cursoragent@cursor.com>
Comment threadsrc/hai_agents_local/manager.py
Comment threadsrc/hai_agents_local/routing.py Outdated
Comment threadsrc/hai_agents_local/sessions.py
… union tag
Co-authored-by: Cursor <cursoragent@cursor.com>

@cursorcursorBot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5dd5a3c. Configure here.

Comment threadsrc/hai_agents/types/environment.py
Co-authored-by: Cursor <cursoragent@cursor.com>
@abonneth
abonneth merged commit dad2bd2 into mainJul 8, 2026
7 checks passed
@abonneth
abonneth deleted the antoine/local-control branch July 8, 2026 16:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@abonneth@adeprezh@cm2435-hcomp