feat(sao): add end-to-end Terminal-Bench SAO with streaming DiLoCo by WaldenLee2005 · Pull Request #1 · agentenv/miles

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

WaldenLee2005 · 2026-08-27T19:04:58Z

Summary

Add hash-bound offline SAO critic value pretraining and mixed-label explained-variance evaluation.
Implement full-parameter online SAO using DIS, observation-skipping length-adaptive GAE, and separate actor/critic optimization.
Connect actor and critic to independent streaming-DiLoCo synchronizers with FP32 updates, fixed quorum, one-GPU islands, and SGLang policy publication.
Add learned trajectory compaction and Codex/OpenEnv Terminal-Bench 2.1 rollouts with managed DinD lifecycle, native verifier evidence, HMAC validation, capacity preflights, and cleanup gates.
Add checkpoint-backed held-out evaluation with publication proof and pass@1-pass@4 reporting.
Document the validated architecture, provenance, results, operating procedure, and limitations.

Production validation

Validated Qwen3.5-0.8B on one 8xH200 node as eight one-GPU islands:

356/356 signed all-task baseline rollouts accepted across all 89 Terminal-Bench 2.1 tasks.
176/176 online-training trajectories accepted, including 42 learned compactions.
Every island completed one actor update and two critic updates without OOM.
Independent actor and critic streaming-DiLoCo sessions reached 8/8 quorum.
All islands agreed on the final actor policy hash.
Critic pretraining loss decreased from 5.8320 to 1.0629.
Corrected mixed-label critic EV was +0.1365 trajectory-weighted and +0.1353 token-weighted, with 7/8 positive batches.
Checkpoint-backed evaluation proved 248/309 served language tensors changed from the base policy.

Matched held-out evaluation showed no aggregate policy improvement:

Metric	Base	Trained
Successful rollouts	1/180	1/180
pass@1	0.5556%	0.5556%
pass@2	1.1111%	1.1111%
pass@3	1.6667%	1.6667%
pass@4	2.2222%	2.2222%

The successful task changed, confirming behavioral change, but this is architecture validation rather than a learning-quality result.

Verification

35/35 production-container tests passed for the corrected evaluation path.
192 focused tests passed after reconciling with current main.
Black, isort, Ruff, Python compilation, and git diff --check passed.
Current main is contained in this branch; there are no merge conflicts.

Caveats

The upstream-integrated tree has not yet been rerun end to end on GPUs; a short production-container GPU smoke remains the next runtime gate.
All 176 online terminal rewards were zero and each island performed only one actor step.
Validation was single-node; physical multi-node Ethernet behavior was not exercised.
The 0.8B experiment did not demonstrate benchmark improvement.

Companion Yeto PR: agentenv/yeto#46.

Artifacts

Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

Allow an external policy synchronizer to keep the rollout loop running until it returns a stop result. Always publish the final trainer weights before stopping and preserve the bounded native loop when the option is disabled.\n\nCover stop-after-publication ordering and repeated trainable-state applies that preserve optimizer state while advancing scheduler progress.

Wake an offloaded training actor before external policy synchronization initializes, keep it resident through the initial rollout weight publication, and offload it only after that publication completes. This lets initial-adapter export and application use live Megatron tensors without leaving the actor awake for the first training step. Extend the external policy sync regression test to assert the exact onload, initialize, publish, offload, train, synchronize, and republish ordering.

Materialize bridge-converted LoRA weights on CPU while the trainer is still resident, before colocated offload releases its model storage. Consume that staged snapshot after rollout weights return, copying only the flattened adapter into a fresh CUDA allocation for CUDA IPC. This avoids both stale offload-backed tensors and CPU file-descriptor serialization across Ray and SGLang process authentication boundaries. Add focused ordering, staging, and IPC transport regressions.

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

AlexEisieand others added 30 commits July 30, 2026 11:30

feat: add external LoRA policy sync boundary
79cbf3e

fix: complete external policy sync reporting
a91bd34

feat: isolate concurrent training ports
5494a6c
Expose deterministic router, rollout-engine, and trainer process-group ports for co-resident Miles drivers. Preserve the requested attention backend when constructing LoRA models through Megatron Bridge.

fix: support gated attention in canonical LoRA
c951c66

build: pin Miles to the patched SGLang revision
a503e41

fix: derive SGLang server host after resolution
a874aa3

build: align Miles with the patched SGLang release
6ab5edf

fix: support mutable SGLang server arguments
4200ff4

fix: pin published SGLang revision
0e161f4

fix: restore SGLang tvm ffi compatibility
a556875

fix: defer p2p update backend import
674498f

feat: support TP/PP external LoRA state
73125d7

fix: sync E288 clone-only grouped expert LoRA
3a83a19

fix: bucket colocated LoRA tensor transport
24cd54c

fix: spill paused trainer state to disk
077dbb9

fix rollout capacity and aborted group replacement
adc54c5

fix: retain collective LoRA state only on rank zero
fa47aec

fix: share external trainable state across actors
0d46c06

fix: keep clone-only policy state sparse
906ab6f

fix: avoid frozen expert CPU scans
5d5342c

fix: avoid redundant r64 state apply scans
8623f8b

fix: offload trainer before initial rollout onload
6bb48b2

fix: reload colocated LoRA base from disk
021ae2d

fix: restore colocated LoRA weights from disk backup
ed9f8c3

feat: support external LoRA sync with pipeline parallelism
c252d87

fix: reclaim LoRA CUDA IPC staging
6062afe

feat(tito): add native Qwen3.8 profile
5a9cf0d

WaldenLee2005 added 10 commits August 19, 2026 22:48

fix(tito): seed Qwen3.8 incremental history
e2ad83d

fix(sglang): support legacy unquantized weight hooks
f31aeb9

fix(megatron): use bridge for distributed full-weight sync
0a62f48

Bind published weight versions to Yeto rollout policy
a8b54de

feat(rl): add SAO value and online training
e25048e

feat(rl): add SAO streaming full-parameter support
d5eb5e8

feat(sao): validate Terminal-Bench streaming DiLoCo e2e
eb33e9e

fix(sao): close production validation gaps
ca1390b

docs(sao): record matched Terminal-Bench validation
79bc3ad

Merge origin/main into feat/sao-tbench21-e2e-validation
16a9bea

WaldenLee2005 mentioned this pull request Aug 27, 2026
feat(rl): add Miles SAO streaming DiLoCo and Terminal-Bench validation agentenv/yeto#46
Merged

WaldenLee2005 merged commit 3c5e19b into mainAug 27, 2026
3 of 5 checks passed

Uh oh!

Conversation

WaldenLee2005 commented Aug 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Production validation

Verification

Caveats

Artifacts

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

WaldenLee2005 commented Aug 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Production validation

Verification

Caveats

Artifacts

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

WaldenLee2005 commented Aug 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Production validation

Verification

Caveats

Artifacts

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

WaldenLee2005 commented Aug 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Production validation

Verification

Caveats

Artifacts

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

WaldenLee2005 commented Aug 27, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WaldenLee2005 commented Aug 27, 2026•
edited
Loading

WaldenLee2005 commented Aug 27, 2026•
edited
Loading

WaldenLee2005 commented Aug 27, 2026•
edited
Loading

WaldenLee2005 commented Aug 27, 2026•
edited
Loading

WaldenLee2005 commented Aug 27, 2026•
edited
Loading

WaldenLee2005 commented Aug 27, 2026•
edited
Loading

WaldenLee2005 commented Aug 27, 2026•
edited
Loading