Skip to content

feat(tasks): add declarative expert program vertical slices - #499

Closed
yuecideng wants to merge 1 commit into
feat/mllm-expert-program-frontendfrom
feat/declarative-task-vertical-slices
Closed

feat(tasks): add declarative expert program vertical slices#499
yuecideng wants to merge 1 commit into
feat/mllm-expert-program-frontendfrom
feat/declarative-task-vertical-slices

Conversation

@yuecideng

@yuecidengyuecideng commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Description

Stack

Migrate repeated Cube Pick/Place and Open Drawer to declarative Expert Programs with typed scene/profile integration. Task classes no longer own waypoint math, invocation assembly, or demo-generation overrides.

The two vertical slices prove configuration-only task expansion when the shared semantic capability already exists, while keeping task validation and physical settling explicit.

Refs #471
Refs #474

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Not applicable.

Validation

  • Focused coverage: tests/gym/envs/expert_program/test_task_vertical_slices.py, cube/open-drawer task tests, and package-data tests
  • Final affected-suite regression on the stack tip: 1215 passed, 2 skipped, 8 deselected
  • Changed Python files pass Black 26.3.1; the Sphinx build and rollout-report drift check pass at the stack tip

Checklist

  • Changed Python files pass Black 26.3.1.
  • Corresponding public/design documentation is included in this stack.
  • Tests cover the affected behavior.
  • No dependency update is required.

@yuecidengyuecideng added task A task written in openai gym format for imitation learning or reinforcement learning gym robot learning env and its related features enhancement New feature or request labels Aug 11, 2026
@yuecideng
yuecideng marked this pull request as ready for review August 11, 2026 16:46
CopilotAI lite review requested due to automatic review settings August 11, 2026 16:46
@greptile-apps

Copy link
Copy Markdown

Greptile Summary

The PR migrates repeated cube pick/place and Open Drawer demonstrations from task-owned motion generation to declarative Expert Programs backed by typed scene and robot-profile bindings.

  • Adds packaged JSON/YAML Expert Programs and selects them from the corresponding Gym configurations.
  • Replaces task-specific trajectory assembly with shared simulation Expert Program adapters.
  • Adds reset settling for the cube task and declarative post-settling and validation behavior.
  • Expands focused task, vertical-slice, and package-data coverage.
  • Updates architectural and user documentation for the semantic runtime, effect monitoring, articulation operations, and Expert Programs.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified.

The declarative program identifiers align with their typed task bindings, configuration paths resolve through the supported loading paths, and the investigated initialization, binding, and test-security concerns did not establish observable failures.

Important Files Changed

FilenameOverview
embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.pyReplaces task-owned pick/place trajectory and validation logic with typed cube scene/profile declarations and the shared Expert Program adapter.
embodichain_tasks/embodichain_tasks/tableware/open_drawer.pyReplaces the fixed drawer-opening trajectory with typed articulation geometry, robot resources, command presets, and shared semantic runtime integration.
embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yamlDeclares three repeated cube pick/place segments with cyclic targets, settling, and target-proximity validation.
embodichain_tasks/configs/expert_program/tableware/open_drawer.jsonDeclares one articulation-operation segment targeting the drawer’s named open state.
embodichain_tasks/configs/gym/multi_segments/cube_pick_place.jsonSelects the packaged cube Expert Program and adds a bounded reset-settling event.
tests/gym/envs/expert_program/test_task_vertical_slices.pyAdds focused coverage for the two declarative task integrations and their shared runtime path.
tests/test_expert_program_package_data.pyVerifies that the added Expert Programs are included and decode correctly from staged package data.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Gym task config] --> B[Packaged Expert Program]
C[Task scene binding] --> D[Simulation Expert Program adapter]
E[Task robot profile binding] --> D
B --> D
D --> F[Semantic compiler and runtime]
F --> G[Buffered commands through env.step]
G --> H[Effect monitoring]
H --> I[Settling and validation metadata]
Loading

Reviews (1): Last reviewed commit: "feat(tasks): add declarative expert prog..." | Re-trigger Greptile

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates two existing simulation tasks (multi-segment cube pick/place and open drawer) from task-owned waypoint/motion planning code to declarative Expert Programs (JSON/YAML) backed by typed scene + robot-profile bindings, validating that tasks can be expanded “configuration-only” when the underlying semantic capability already exists.

Changes:

  • Convert MultiSegmentsCubePickPlaceEnv and OpenDrawerEnv to use ExpertProgramEnvironmentMixin + create_simulation_expert_program_adapter, leaving tasks responsible only for scene/profile declarations.
  • Add packaged Expert Program resources (repeated_cube_pick_place.yaml, open_drawer.json) and update Gym configs to reference them via expert_program_path.
  • Add focused vertical-slice tests and update documentation/context to cover Expert Programs, effect evidence/monitoring, and articulation operation semantics.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/test_expert_program_package_data.pyVerifies setuptools stages Expert Program JSON/YAML and that staged resources decode via installed config paths.
tests/gym/envs/tasks/test_open_drawer.pyValidates OpenDrawer task registration, config wiring to packaged program, adapter construction, and (optional) real-sim execution evidence.
tests/gym/envs/tasks/test_multi_segments_cube_pick_place.pyValidates cube task registration, config wiring to packaged program, adapter construction, and profile/settle configuration.
tests/gym/envs/expert_program/test_task_vertical_slices.pyEnd-to-end non-physical vertical slices: strict decode/compile, lazy lifecycles, settle/validator metadata, and task “no motion override” guarantees.
embodichain_tasks/embodichain_tasks/tableware/open_drawer.pyReplaces task-local motion generation with typed scene/profile declarations for operate_articulation.
embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.pyReplaces task-local lazy planning/settling with typed scene/profile declarations plus packaged repeated-cube Expert Program.
embodichain_tasks/configs/gym/open_drawer/cobot_magic_3cam.jsonAdds expert_program_path to select the packaged drawer program.
embodichain_tasks/configs/gym/multi_segments/cube_pick_place.jsonAdds expert_program_path, moves settling to reset events, and trims extensions to semantic-only knobs.
embodichain_tasks/configs/expert_program/tableware/open_drawer.jsonAdds the declarative Open Drawer Expert Program (segment invoking operate_articulation).
embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yamlAdds the declarative repeated cube pick/place Expert Program (repeat of segment with post-policy + validator).
docs/source/tutorial/atomic_actions.rstDocuments scene-dependency monitoring cutoffs via scene_dependency_monitor_until.
docs/source/overview/sim/index.rstAdds navigation guidance for when to use Expert Programs.
docs/source/overview/sim/atomic_actions/robot_skill_profiles.mdExtends docs for effect monitors, endpoint metadata, and Expert Program integration patterns.
docs/source/overview/sim/atomic_actions/index.mdAdds Expert Programs doc to the atomic-actions overview and clarifies effect evidence vs decision responsibilities.
docs/source/overview/sim/atomic_actions/builtin_actions.mdDocuments operate_articulation and its dependency-monitor cutoff semantics.
docs/design/declarative_expert_program_plan.mdUpdates project design plan status/progress notes for the stacked implementation.
agent_context/topics/atomic-actions/atomic-actions.mdUpdates agent-facing context with semantic runtime + Expert Program details.
agent_context/MAP.yamlExpands topic keywords and file pointers for Expert Program / semantic runtime content.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +179 to +181
environment = os.environ.copy()
environment["PYTHONPATH"] = str(staged_config_package.build_lib)
completed = subprocess.run(
@yuecideng

Copy link
Copy Markdown
ContributorAuthor

Folded into #500 during stacked-PR consolidation. Its commits remain included in #500; the remote branch is retained for traceability.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancementNew feature or requestgymrobot learning env and its related featurestaskA task written in openai gym format for imitation learning or reinforcement learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@yuecideng