Skip to content

feat(sim): add semantic runtime effects and parallelism - #496

Merged
yuecideng merged 40 commits into
mainfrom
feat/semantic-skill-runtime
Aug 22, 2026
Merged

feat(sim): add semantic runtime effects and parallelism#496
yuecideng merged 40 commits into
mainfrom
feat/semantic-skill-runtime

Conversation

@yuecideng

@yuecidengyuecideng commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Description

Stack

Parallel execution remains fail closed: resource analysis and shared-clock coordination are implemented, but production concurrency still requires an authoritative physical safety validator.

Refs #471
Refs #474

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Not applicable.

Validation

  • Focused coverage: tests/sim/skills, including effects, evidence, runtime, parallel compiler/runtime, and conditional cuRobo recovery coverage
  • Final affected-suite regression on the stack tip: 1215 passed, 2 skipped, 8 deselected
  • Changed Python files pass Black 26.3.1; the Sphinx build and rollout-report drift check pass at the stack tip

Checklist

  • Changed Python files pass Black 26.3.1.
  • Corresponding public/design documentation is included in this stack.
  • Tests cover the affected behavior.
  • No dependency update is required.

@yuecidengyuecideng added atomic action atomic action related functionality enhancement New feature or request labels Aug 11, 2026
@yuecideng
yuecideng marked this pull request as ready for review August 11, 2026 16:46
CopilotAI lite review requested due to automatic review settings August 11, 2026 16:46

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds the canonical semantic-skill runtime layer (effects + evidence collection + runtime traces) and introduces a strict, versioned parallel-execution contract that compiles down to the existing atomic-action engine. This extends the simulation “skills” facade with first-class articulation semantics, effect-monitor configuration via policy presets, and deterministic metadata/tracing for downstream program/runtime layers.

Changes:

  • Extend the scene/registry model to publish canonical articulation joint observations and evidence addresses.
  • Add effect-monitor + evidence-source plumbing into robot skill profiles/presets and semantic integration validation.
  • Introduce deterministic parallel timing/state-merge contracts plus extensive test coverage (including a GPU cuRobo dynamic-collision recovery gate).

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
tests/sim/skills/test_scene.pyAdds joint-observation registration/snapshot behavior tests and simulation alias semantics coverage.
tests/sim/skills/test_runtime.pyNew end-to-end tests for SkillRuntime behavior, tracing/metadata, row masking, and parallel factory interactions.
tests/sim/skills/test_profiles.pyValidates endpoint effect-source snapshotting and preset effect-monitor behavior.
tests/sim/skills/test_parallel.pyNew tests for parallel alignment, claim conflicts, timing validation, and deterministic barrier/state merging.
tests/sim/skills/test_integration.pyAdds binding-time validation for safe preset dynamic-collision requirements and preset reachability semantics.
tests/sim/skills/test_evidence.pyNew tests for evidence providers/collector registry, query building, and articulation evidence provider behavior.
tests/sim/skills/test_effects.pyNew tests for effect contracts, monitor refs/registries, and composite monitor behavior.
tests/sim/skills/test_curobo_semantic_runtime_dynamic_recovery_gpu.pyGPU/cuRobo integration test ensuring replanning on dynamic collision-world updates.
tests/sim/skills/test_compiler.pyExtends compiler tests for effect monitor selection/validation and dynamic collision policy propagation.
tests/sim/skills/test_calls.pyAdds deterministic, JSON-safe semantic-call metadata serialization test.
tests/sim/skills/test_articulation_semantics.pyNew tests for OperateArticulation call semantics, grounding, and typed articulation effects.
embodichain/lab/sim/skills/scene.pyAdds articulation joint evidence address + joint-state providers and integrates joint observations into scene versioning.
embodichain/lab/sim/skills/profiles.pyAdds endpoint effect-sources, task-state key plumbing, and preset-owned effect-monitor selections.
embodichain/lab/sim/skills/parallel.pyNew deterministic parallel execution contracts (claims, timing grid, effect merge, barrier resolution).
embodichain/lab/sim/skills/integration.pyAdds OperateArticulation call linking and safe dynamic-collision policy preflight validation.
embodichain/lab/sim/skills/calls.pyAdds call metadata serialization and introduces OperateArticulation semantic call type.
embodichain/lab/sim/skills/init.pyExposes new runtime/effects/evidence/parallel APIs through package exports.
docs/source/overview/sim/atomic_actions/robot_skill_profiles.mdDocuments safe preset dynamic-collision rules and preset-driven effect monitor selection.
docs/source/api_reference/embodichain/embodichain.lab.sim.skills.rstExpands API reference to include semantic runtime, effects/evidence, and parallel execution types.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +1993 to +2002
if not torch.equal(
current_position, previous_position
) or not torch.equal(
current_validity,
previous_validity,
):
changed = True
break
self._store_joint_baseline(states)
return changed
@greptile-apps

greptile-appsBot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds a canonical semantic-skill runtime with typed physical-effect verification and a fail-closed parallel execution foundation.

  • Replaces the earlier semantic task facade with per-call grounding, execution, evidence collection, tracing, and verified state transitions.
  • Adds typed effect specifications, evidence providers, composite monitors, and row-local hysteresis.
  • Adds resource and symbolic-write analysis, shared-clock branch coordination, command safety validation, and deterministic parallel state merging.
  • Updates atomic-action primitives, public exports, documentation, and focused tests for the new contracts.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope.

No blocking failure remains.

Important Files Changed

FilenameOverview
embodichain/lab/sim/skills/runtime.pyIntroduces the canonical sequential semantic runtime, per-call execution lifecycle, verified state propagation, structured failures, and compatibility facade.
embodichain/lab/sim/skills/effects.pyAdds typed semantic effects, evidence clauses, monitor selection, composite evaluation, and row-local hysteresis.
embodichain/lab/sim/skills/evidence.pyAdds versioned evidence routing, collection contexts, provider contracts, and backend-neutral physical observations.
embodichain/lab/sim/skills/parallel.pyDefines deterministic resource analysis, command alignment, barrier resolution, and symbolic state-merge rules.
embodichain/lab/sim/skills/parallel_runtime.pyAdds fail-closed shared-clock coordination for forked semantic lanes with mandatory outbound-command safety validation.
embodichain/lab/sim/skills/compiler.pyExtends semantic analysis and grounding to produce typed effect specifications, monitors, and parallel-analysis metadata.
embodichain/lab/sim/atomic_actions/state.pyExtends row-local symbolic task state and state-delta application for verified runtime effects.
tests/sim/skills/test_parallel_runtime.pyAdds focused coverage for synchronization, timeout, cancellation, safe-stop acknowledgement, command validation, and parallel state merging.

Sequence Diagram

sequenceDiagram
participant App
participant Runtime as SkillRuntime
participant Compiler as SemanticSkillCompiler
participant Runner as ExecutionRunner
participant Evidence as EffectEvidenceCollector
participant Monitor as EffectMonitor
App->>Runtime: start semantic calls
Runtime->>Compiler: analyze complete call window
loop Each executable call
Runtime->>Compiler: ground using fresh PlanningContext
Runtime->>Runner: execute atomic invocation
Runner-->>Runtime: request effect verification
Runtime->>Evidence: collect typed physical evidence
Evidence-->>Monitor: correlated row evidence
Monitor-->>Runtime: success/failure masks
Runtime->>Runtime: commit verified StateDelta
end
Runtime-->>App: SkillResult and traces
Loading

Reviews (6): Last reviewed commit: "wip" | Re-trigger Greptile

Make endpoint bindings, timed command frames, transports, routing, safe holds, and profile lowering controller-agnostic. Preserve joint trajectories as optional feedback artifacts and add staged, same-address invocation revision semantics for mobile and whole-body safety.
…2c-runtime-endpoints
# Conflicts:
#	docs/design/declarative_expert_program_plan.md
#	embodichain/lab/sim/atomic_actions/primitives/coordinated_pickment.py
#	embodichain/lab/sim/atomic_actions/primitives/coordinated_placement.py
#	embodichain/lab/sim/atomic_actions/primitives/hand_over.py
#	embodichain/lab/sim/atomic_actions/primitives/move_end_effector.py
#	embodichain/lab/sim/atomic_actions/primitives/move_held_object.py
#	embodichain/lab/sim/atomic_actions/primitives/move_joints.py
#	embodichain/lab/sim/atomic_actions/primitives/pick_up.py
#	embodichain/lab/sim/atomic_actions/primitives/place.py
#	embodichain/lab/sim/atomic_actions/primitives/press.py
#	embodichain/lab/sim/skills/profiles.py
#	scripts/tutorials/atomic_action/assemble.py
#	scripts/tutorials/atomic_action/coordinated_pickment.py
#	scripts/tutorials/atomic_action/coordinated_placement.py
#	scripts/tutorials/atomic_action/hand_over.py
#	scripts/tutorials/atomic_action/move_end_effector.py
#	scripts/tutorials/atomic_action/move_held_object.py
#	scripts/tutorials/atomic_action/move_joints.py
#	scripts/tutorials/atomic_action/place.py
#	scripts/tutorials/atomic_action/press.py
Remove fallback control periods from motion policies and the atomic action engine. Require planners and custom actions to provide validated timing, propagate explicit interpolation cadence through planning contexts, and update built-ins, tasks, benchmarks, tutorials, docs, and tests.
BREAKING CHANGE: planner results with positions must include dt and duration, custom atomic actions must return TimedTrajectory, and fallback_control_dt is removed.
…pr2c-runtime-endpoints
# Conflicts:
#	agent_context/topics/atomic-actions/atomic-actions.md
#	docs/source/overview/sim/atomic_actions/builtin_actions.md
#	docs/source/overview/sim/atomic_actions/index.md
#	docs/source/tutorial/atomic_actions.rst
#	embodichain/lab/sim/atomic_actions/execution.py
#	scripts/tutorials/atomic_action/dynamic_obstacle_recovery.py
#	tests/sim/atomic_actions/test_core.py
#	tests/sim/atomic_actions/test_engine.py
#	tests/sim/atomic_actions/test_runner.py
…2c-runtime-endpoints
# Conflicts:
#	agent_context/topics/atomic-actions/atomic-actions.md
#	agent_context/topics/motion-planning/motion-planning.md
#	docs/design/declarative_expert_program_plan.md
#	docs/source/overview/sim/atomic_actions/builtin_actions.md
#	docs/source/overview/sim/atomic_actions/index.md
#	docs/source/overview/sim/planners/motion_generator.md
#	docs/source/tutorial/atomic_actions.rst
#	docs/source/tutorial/motion_gen.rst
#	embodichain/lab/sim/atomic_actions/core.py
#	embodichain/lab/sim/atomic_actions/execution.py
#	embodichain/lab/sim/atomic_actions/policies.py
#	embodichain/lab/sim/atomic_actions/primitives/coordinated_pickment.py
#	embodichain/lab/sim/atomic_actions/primitives/coordinated_placement.py
#	embodichain/lab/sim/atomic_actions/primitives/hand_over.py
#	embodichain/lab/sim/atomic_actions/primitives/move_end_effector.py
#	embodichain/lab/sim/atomic_actions/primitives/move_held_object.py
#	embodichain/lab/sim/atomic_actions/primitives/move_joints.py
#	embodichain/lab/sim/atomic_actions/primitives/pick_up.py
#	embodichain/lab/sim/atomic_actions/primitives/place.py
#	embodichain/lab/sim/atomic_actions/primitives/press.py
#	embodichain/lab/sim/planners/base_planner.py
#	embodichain/lab/sim/planners/motion_generator.py
#	embodichain/lab/sim/planners/neural_planner.py
#	embodichain/lab/sim/planners/utils.py
#	scripts/benchmark/atomic_action/press_benchmark.py
#	scripts/benchmark/motion_generation/planners/ik_interpolate.py
#	scripts/tutorials/atomic_action/control_dt.py
#	scripts/tutorials/atomic_action/dynamic_obstacle_recovery.py
#	scripts/tutorials/atomic_action/press.py
#	tests/benchmark/motion_generation/test_motion_generation_benchmark.py
#	tests/sim/atomic_actions/test_actions.py
#	tests/sim/atomic_actions/test_core.py
#	tests/sim/atomic_actions/test_engine.py
#	tests/sim/atomic_actions/test_runner.py
#	tests/sim/planners/test_motion_generator_batched.py
#	tests/sim/planners/test_plan_state_batched.py
…mantic-skill-compiler
# Conflicts:
#	agent_context/MAP.yaml
#	agent_context/topics/atomic-actions/atomic-actions.md
#	agent_context/topics/motion-planning/motion-planning.md
#	docs/design/declarative_expert_program_plan.md
#	docs/source/overview/sim/atomic_actions/builtin_actions.md
#	docs/source/overview/sim/atomic_actions/index.md
#	docs/source/overview/sim/atomic_actions/robot_skill_profiles.md
#	docs/source/tutorial/atomic_actions.rst
#	embodichain/lab/sim/atomic_actions/__init__.py
#	embodichain/lab/sim/atomic_actions/affordance.py
#	embodichain/lab/sim/atomic_actions/bindings.py
#	embodichain/lab/sim/atomic_actions/core.py
#	embodichain/lab/sim/atomic_actions/effects.py
#	embodichain/lab/sim/atomic_actions/engine.py
#	embodichain/lab/sim/atomic_actions/execution.py
#	embodichain/lab/sim/atomic_actions/plans.py
#	embodichain/lab/sim/atomic_actions/primitives/coordinated_pickment.py
#	embodichain/lab/sim/atomic_actions/primitives/coordinated_placement.py
#	embodichain/lab/sim/atomic_actions/primitives/hand_over.py
#	embodichain/lab/sim/atomic_actions/primitives/move_end_effector.py
#	embodichain/lab/sim/atomic_actions/primitives/move_held_object.py
#	embodichain/lab/sim/atomic_actions/primitives/move_joints.py
#	embodichain/lab/sim/atomic_actions/primitives/pick_up.py
#	embodichain/lab/sim/atomic_actions/primitives/place.py
#	embodichain/lab/sim/atomic_actions/primitives/press.py
#	embodichain/lab/sim/atomic_actions/state.py
#	embodichain/lab/sim/planners/base_planner.py
#	embodichain/lab/sim/planners/curobo/curobo_planner.py
#	embodichain/lab/sim/planners/curobo/curobo_yaml.py
#	embodichain/lab/sim/planners/motion_generator.py
#	embodichain/lab/sim/skills/__init__.py
#	embodichain/lab/sim/skills/profiles.py
#	embodichain/lab/sim/skills/scene.py
#	scripts/benchmark/atomic_action/press_benchmark.py
#	scripts/tutorials/atomic_action/assemble.py
#	scripts/tutorials/atomic_action/coordinated_pickment.py
#	scripts/tutorials/atomic_action/coordinated_placement.py
#	scripts/tutorials/atomic_action/dynamic_obstacle_recovery.py
#	scripts/tutorials/atomic_action/hand_over.py
#	scripts/tutorials/atomic_action/move_end_effector.py
#	scripts/tutorials/atomic_action/move_held_object.py
#	scripts/tutorials/atomic_action/move_joints.py
#	scripts/tutorials/atomic_action/moving_target_recovery.py
#	scripts/tutorials/atomic_action/pickup.py
#	scripts/tutorials/atomic_action/place.py
#	scripts/tutorials/atomic_action/press.py
#	tests/sim/atomic_actions/test_actions.py
#	tests/sim/atomic_actions/test_control.py
#	tests/sim/atomic_actions/test_core.py
#	tests/sim/atomic_actions/test_endpoint_runtime_e2e.py
#	tests/sim/atomic_actions/test_engine.py
#	tests/sim/atomic_actions/test_runner.py
#	tests/sim/atomic_actions/test_trajectory_ops.py
#	tests/sim/planners/test_curobo_planner.py
#	tests/sim/planners/test_motion_generator_batched.py
#	tests/sim/skills/test_profiles.py
#	tests/sim/skills/test_scene.py
@skywhite1024
skywhite1024force-pushed the feat/semantic-skill-runtime branch from 974368f to 370e5f8CompareAugust 21, 2026 09:35
@skywhite1024
skywhite1024force-pushed the feat/semantic-skill-runtime branch from 370e5f8 to 974368fCompareAugust 21, 2026 11:31
…rified-runtime
# Conflicts:
#	agent_context/topics/atomic-actions/atomic-actions.md
#	docs/source/overview/sim/atomic_actions/index.md
#	docs/source/overview/sim/semantic_skills.md
#	docs/source/tutorial/atomic_actions.rst
#	docs/source/tutorial/semantic_skills.rst
#	embodichain/lab/sim/atomic_actions/core.py
#	embodichain/lab/sim/atomic_actions/execution.py
#	embodichain/lab/sim/atomic_actions/plans.py
#	embodichain/lab/sim/atomic_actions/primitives/pick_up.py
#	embodichain/lab/sim/skills/runtime.py
#	tests/sim/atomic_actions/test_actions.py
#	tests/sim/atomic_actions/test_engine_per_env.py
#	tests/sim/skills/test_runtime.py
Base automatically changed from feat/atomic-action-verified-runtime to mainAugust 21, 2026 16:28
@yuecideng
yuecideng merged commit df06818 into mainAug 22, 2026
4 of 5 checks passed
@yuecideng
yuecideng deleted the feat/semantic-skill-runtime branch August 22, 2026 15:42
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

atomic actionatomic action related functionalityenhancementNew feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@yuecideng