[BugFix] Reject invalid ReplayBuffer generations

[theap06]

Rejects never-written replay slots from update_if_present when both generation values are -1.

Test: pytest test/rb/test_rb_core.py -k TestUpdateIfPresent

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4043

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 1a289eb with merge base 82a2593 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[theap06]

Rebased onto current main and #4046, resolved the obsolete generation-stamp history, and marked this ready for review. The focused core, Ray, and doctest coverage passes locally. I do not have permission to request reviewers or apply labels; @vmoens, could you review and add the ci/optdeps label, then rerun CI after the label is present? This PR touches the Ray optional-dependency integration.

[vmoens]

Reviewed rebased head b8bf0a7a0bc3ec5b995569aaf1414f86ac2553b7, including the latest generation-stamp reconciliation commit and the request for optional-dependency CI.

One blocking correctness issue is at torchrl/data/replay_buffers/replay_buffers.py in the live = current == generation comparison. Generation -1 explicitly means “no usable stamp”, but equality treats (-1, -1) as live. I reproduced this with a size-10 lazy storage containing only 5 records: update_if_present(index=tensor([7]), generation=tensor([-1]), patch={"x": tensor([99.])}) returns updated=True and writes slot 7 even though len(rb) == 5. Please exclude negative current/captured generations (and add the never-written-slot regression) so only genuine live handles can mutate storage.

Targeted result: all 14 TestUpdateIfPresent tests pass, which confirms the missing case. After the fix, the author-requested ci/optdeps label/rerun is still appropriate because the diff changes the Ray integration.

[theap06]

Since #4049 landed the conditional-update feature on main, I rebased this PR down to the reported correctness fix. Negative current or captured generations are now always stale, with a never-written-slot regression. Focused result: 30 passed.

[vmoens]

Just a lint issue

[vmoens]

LGTM