Skip to content

Updated Cosmos3 client to work with vllm-omni server - #26

Open
MaciejBalaNV wants to merge 2 commits into
NVlabs:mainfrom
MaciejBalaNV:cosmos3_vllm_omni_update
Open

Updated Cosmos3 client to work with vllm-omni server#26
MaciejBalaNV wants to merge 2 commits into
NVlabs:mainfrom
MaciejBalaNV:cosmos3_vllm_omni_update

Conversation

@MaciejBalaNV

@MaciejBalaNVMaciejBalaNV commented Aug 21, 2026

Copy link
Copy Markdown

Description

This PR adds a support for a RoboLab client in vllm-omni. We achieve this by:

  1. Added MsgPackWebSocketTransport that handles metadata handshake and auth.
  2. Added remote-uri and remote-token - first one is important for choosing the endpoint, second one is important for auth.
  3. Correctly parsing the response

Additionally, did some refactoring of the DreamZero model to avoid duplication of the MsgPack code.

On vllm-omni side this PR also needs to be merged: vllm-project/vllm-omni#6460

PR type

  • New asset (object / scene / task / robot / variation)
  • Bug fix
  • New policy backend (policies/<name>/) — requires a paper/project reference and prior contact with the maintainers (see CONTRIBUTING.md)
  • Documentation
  • Other (describe above)

Checklist

  • All commits are signed off (git commit -s) — see DCO
  • New Python files carry the standard SPDX header (Apache-2.0, NVIDIA copyright — copy from any existing file)
  • New assets are self-contained in their own folder, include their license, and are listed in THIRD_PARTY_NOTICES.md
  • Binary files (.usd, meshes, textures, images) are tracked with git-LFS
  • uv run pytest tests/ passes locally
  • New robots: env configs compile against the benchmark tasks (see docs/robots.md)

Asset attribution

Signed-off-by: Maciej Bala <mbala@nvidia.com>
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@MaciejBalaNV