feat(tasks): add declarative task slices and rollout validation

[yuecideng]

Description

Stack

• Layer: 8/14
• Base: feat/mllm-expert-program-frontend
• Depends on: feat(agents): add strict expert program frontend #498
• Followed by: refactor(atomic-actions): add typed tracking contracts #501

Consolidation

Consolidates #499 into this layer. Scope: declarative task vertical slices and deterministic rollout validation.
Add deterministic Expert Program rollout measurement: fixed-seed execution, raw trial JSON, failure-preserving aggregation, Markdown reports, static migration-size checks, and supported-simulation gates.

Benchmark reporting distinguishes framework coverage, checked-in integrations, and physical acceptance instead of treating compilation success as task success.

Refs #471
Refs #474

Type of change

• Enhancement (non-breaking change which improves existing functionality)

Screenshots

Not applicable.

Validation

• Focused coverage: tests/benchmark/expert_program/test_demo_success.py and tests/scripts/tools/test_expert_program_rollout_report.py
• Final affected-suite regression on the stack tip: 1215 passed, 2 skipped, 8 deselected
• Changed Python files pass Black 26.3.1; the Sphinx build and rollout-report drift check pass at the stack tip

Checklist

• Changed Python files pass Black 26.3.1.
• Corresponding public/design documentation is included in this stack.
• Tests cover the affected behavior.
• No dependency update is required.

[greptile-apps]

Greptile Summary

The PR consolidates two task integrations onto the declarative Expert Program runtime and adds deterministic rollout measurement and reporting.

• Removes the legacy repeated-pick/place and drawer task implementations and configurations.
• Adds fixed-seed demo execution, raw trial artifacts, failure-preserving aggregation, and Markdown reporting.
• Adds migration-size and capability snapshots plus focused package, task, and simulation tests.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope.

No blocking failure remains.

Important Files Changed

Filename	Overview
scripts/benchmark/expert_program/demo_success.py	Adds fixed-seed, no-retry demo execution with failure-preserving raw artifacts and deterministic aggregation.
scripts/tools/expert_program_rollout_report.py	Generates and checks the static capability and migration-size report from explicit source snapshots.
embodichain_tasks/embodichain_tasks/expert_program/repeated_pick_place.py	Simplifies the canonical repeated-pick environment by removing its task-local contact observer while explicitly withholding physical acceptance.
embodichain_tasks/configs/gym/expert_program/repeated_pick_place.json	Adds dataset recording metadata and removes the contact sensor from the structurally supported but physically unqualified task.
tests/benchmark/expert_program/test_demo_success.py	Provides broad pure-Python coverage for deterministic execution, failures, aggregation, serialization, and cleanup behavior.
tests/test_expert_program_package_data.py	Verifies Expert Program resources are included and decodable from staged package output.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Fixed benchmark seeds] --> B[Seeded non-saving reset]
B --> C[Execute declarative demo once]
C --> D[Normalize per-environment results]
D --> E[Non-saving discard reset]
E --> F[Raw JSON trials]
F --> G[Aggregate metrics]
G --> H[Markdown report]

Loading

_{Reviews (5): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.