Skip to content

Add outcome-aware dataset fragments - #547

Draft
yuecideng wants to merge 1 commit into
mainfrom
codex/segment-outcome-resume
Draft

Add outcome-aware dataset fragments#547
yuecideng wants to merge 1 commit into
mainfrom
codex/segment-outcome-resume

Conversation

@yuecideng

Copy link
Copy Markdown
Contributor

Description

This PR adds stable per-segment outcome and provenance annotations to demonstration execution and lets collectors persist eligible program segments as independent LeRobot episodes.

The change preserves successful work from partially failed expert programs without conflating segment quality with episode termination. The storage design follows the current LeRobot Dataset v3 episode/task model, while using EmbodiChain-owned names such as segment_accepted instead of overloading policy/evaluation fields such as next.success.

Key changes:

  • add DemoExecutionCfg with continuous and segment_fragments persistence modes;
  • classify each segment row by its first authoritative failure phase and record dense attempt, acceptance, and continuity annotations;
  • split accepted segments into independent episodes in synchronous and asynchronous LeRobot recorders, with failed-fragment persistence as an explicit opt-in;
  • prevent online segment/boundary sampling from selecting rejected frames or crossing continuity boundaries;
  • propagate retry attempt IDs through online generation and the CLI collection path;
  • update the design status, public API docs, and focused regression coverage.

Checkpoint capture, restore, and resume are intentionally deferred until an authoritative restore port exists. This PR records continuity_id for forward compatibility, with current executions remaining in continuity region zero.

Design: segment outcome and resume plan

LeRobot references: Datasets v3 overview, current dataset API

Dependencies: None. The existing LeRobot version constraint is unchanged.

Issue: None.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (non-breaking change which improves an existing functionality)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (existing functionality will not work without user modification)
  • Documentation update

Screenshots

Not applicable; this change has no visual UI.

Validation

  • black . — 748 files unchanged
  • python docs/scripts/check_api_docs.py — 1427/1427 exports documented
  • python -m pytest -q tests/gym/envs tests/data_pipeline tests/lab/scripts/test_run_env.py tests/gym/utils/test_gym_utils.py -m "not requires_sim and not gpu" --disable-warnings --maxfail=20 — 680 passed, 1 skipped, 22 deselected
  • python -m pytest -q tests/docs --confcutdir=tests/docs --disable-warnings — 20 passed
  • git diff --check — passed

The full simulator/GPU suite was not run locally; those markers were excluded from the proportional affected-area test run.

Checklist

  • I have run the black . command to format the code base.
  • I have made corresponding changes to the documentation.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py).
  • I have added tests that prove the feature works.
  • Dependencies have been updated, if applicable (not applicable; no dependency changes).

Add stable per-segment outcomes and dense provenance annotations, persist accepted segments as independent LeRobot episodes, and constrain online sampling by acceptance and continuity. Checkpoint capture and resume remain intentionally deferred.
@yuecidengyuecideng added enhancement New feature or request dataset gym robot learning env and its related features data Related to data_pipeline module labels Aug 24, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dataRelated to data_pipeline moduledatasetenhancementNew feature or requestgymrobot learning env and its related features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@yuecideng