Uh oh!
There was an error while loading. Please reload this page.
feat(agents): add strict expert program frontend - #498
Conversation
Make endpoint bindings, timed command frames, transports, routing, safe holds, and profile lowering controller-agnostic. Preserve joint trajectories as optional feedback artifacts and add staged, same-address invocation revision semantics for mobile and whole-body safety.
Greptile SummaryThe PR adds a strict MLLM-facing frontend that injects host-owned integration configuration, decodes constrained Expert Program JSON, enforces sequential and curated-call policies, and delegates compilation through the existing environment adapter.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| embodichain/agents/mllm/expert_program.py | Adds strict decoding, policy enforcement, trusted integration injection, and compilation delegation without an eligible follow-up defect. |
| tests/agents/mllm/test_expert_program.py | Covers integration ownership, schema and node restrictions, resource overrides, strict JSON failures, and adapter compilation. |
| tests/gym/envs/expert_program/test_simulation_environment.py | Extends runtime-equivalence coverage to confirm that MLLM, decoded configuration, and Python skill frontends produce equivalent invocations and results. |
| embodichain/agents/mllm/init.py | Exports the two new MLLM Expert Program entry points. |
| docs/source/api_reference/embodichain/embodichain.lab.gym.envs.expert_program.rst | Documents the constrained sequential MLLM frontend and its public functions. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
M[MLLM JSON response] --> P[Strict JSON parser]
H[Host integration] --> I[Inject trusted integration]
P --> I
I --> D[Canonical Expert Program decoder]
D --> V[MLLM policy validation]
V --> A[Environment adapter compile]
A --> C[Provider-free compiled program]
Reviews (5): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile
There was a problem hiding this comment.
Pull request overview
Adds a strict MLLM-facing Expert Program frontend under embodichain.agents.mllm that (a) injects host-owned integration, (b) restricts the model to a curated schema-v1 surface, and (c) compiles via the existing Expert Program adapter/compiler path so model, config, and Python callers share one validation+lowering pipeline.
Changes:
- Introduces
decode_mllm_expert_program()/compile_mllm_expert_program()with a narrow policy (schema v1 only, curated call types only, no resource overrides, host-injected integration). - Adds focused unit tests for MLLM decode/compile behavior plus an integration-style equivalence test alongside existing simulation fixtures.
- Updates API reference docs to expose the new MLLM frontend functions.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/gym/envs/expert_program/test_simulation_environment.py | Extends simulation fixture tests to assert equivalence across Python calls, config decoding, and the MLLM frontend. |
| tests/agents/mllm/test_expert_program.py | New unit tests covering MLLM integration injection, strict policy rejection cases, strict JSON parsing behavior, and compile-time scene reference errors. |
| embodichain/agents/mllm/expert_program.py | Implements strict MLLM decode + compile frontend that reuses existing Expert Program parsing/decoding/validation and adapter compilation. |
| embodichain/agents/mllm/init.py | Exposes the MLLM frontend functions as the public embodichain.agents.mllm API surface. |
| embodichain/agents/init.py | Introduces the embodichain.agents package initializer for agent-facing frontends. |
| docs/source/api_reference/embodichain/embodichain.lab.gym.envs.expert_program.rst | Documents and publishes the MLLM frontend functions in the API reference. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| Raises: | ||
| TypeError: If ``adapter`` or ``integration`` has the wrong exact type. | ||
| ExpertProgramDecodeError: If JSON, schema, or MLLM policy validation | ||
| fails. | ||
| """ |
| Raises: | ||
| TypeError: If ``integration`` is not an exact integration config. | ||
| ExpertProgramDecodeError: If JSON, schema, or MLLM policy validation | ||
| fails. | ||
| """ |
Make endpoint bindings, timed command frames, transports, routing, safe holds, and profile lowering controller-agnostic. Preserve joint trajectories as optional feedback artifacts and add staged, same-address invocation revision semantics for mobile and whole-body safety.
…2c-runtime-endpoints # Conflicts: # docs/design/declarative_expert_program_plan.md # embodichain/lab/sim/atomic_actions/primitives/coordinated_pickment.py # embodichain/lab/sim/atomic_actions/primitives/coordinated_placement.py # embodichain/lab/sim/atomic_actions/primitives/hand_over.py # embodichain/lab/sim/atomic_actions/primitives/move_end_effector.py # embodichain/lab/sim/atomic_actions/primitives/move_held_object.py # embodichain/lab/sim/atomic_actions/primitives/move_joints.py # embodichain/lab/sim/atomic_actions/primitives/pick_up.py # embodichain/lab/sim/atomic_actions/primitives/place.py # embodichain/lab/sim/atomic_actions/primitives/press.py # embodichain/lab/sim/skills/profiles.py # scripts/tutorials/atomic_action/assemble.py # scripts/tutorials/atomic_action/coordinated_pickment.py # scripts/tutorials/atomic_action/coordinated_placement.py # scripts/tutorials/atomic_action/hand_over.py # scripts/tutorials/atomic_action/move_end_effector.py # scripts/tutorials/atomic_action/move_held_object.py # scripts/tutorials/atomic_action/move_joints.py # scripts/tutorials/atomic_action/place.py # scripts/tutorials/atomic_action/press.py
2df667f to
8bf6a50Compare8bf6a50 to
2df667fCompare…rified-runtime # Conflicts: # agent_context/topics/atomic-actions/atomic-actions.md # docs/source/overview/sim/atomic_actions/index.md # docs/source/overview/sim/semantic_skills.md # docs/source/tutorial/atomic_actions.rst # docs/source/tutorial/semantic_skills.rst # embodichain/lab/sim/atomic_actions/core.py # embodichain/lab/sim/atomic_actions/execution.py # embodichain/lab/sim/atomic_actions/plans.py # embodichain/lab/sim/atomic_actions/primitives/pick_up.py # embodichain/lab/sim/skills/runtime.py # tests/sim/atomic_actions/test_actions.py # tests/sim/atomic_actions/test_engine_per_env.py # tests/sim/skills/test_runtime.py
2df667f to
0b356f9Compare…hub.com/DexForce/EmbodiChain into feat/declarative-expert-program-runtime
…-expert-program-frontend # Conflicts: # docs/source/api_reference/embodichain/embodichain.lab.gym.envs.expert_program.rst # tests/gym/envs/expert_program/test_simulation_environment.py
…ram-frontend # Conflicts: # docs/source/api_reference/embodichain/embodichain.lab.gym.envs.expert_program.rst # docs/source/api_reference/public_api.rst # embodichain/toolkits/graspkit/pg_grasp/__init__.py # tests/gym/envs/expert_program/test_simulation_environment.py
Uh oh!
There was an error while loading. Please reload this page.
Description
Stack
feat/declarative-expert-program-runtimeAdd an MLLM-facing Expert Program frontend that renders the exact integration catalog into prompts and decodes model output through the same strict schema and preflight path used by files and Python callers.
The frontend cannot invent unregistered calls, providers, scene references, or low-level controller commands, preserving one semantic facade from generation through execution.
Refs #471
Refs #474
Type of change
Screenshots
Not applicable.
Validation
pytest -q tests/agents/mllm/test_expert_program.py— 21 passedChecklist