[DLRMv4] Restrict the dlrmv4 learning rate to a band around the scaled reference by chriscai-amd · Pull Request #473 · mlcommons/logging

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd · 2026-08-31T09:20:48Z

The closed division left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it.

Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the check guards on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The sparse learning rate is already pinned to the dense one, so the band applies to both channels.

The open division is deliberately untouched: it exists to permit recipe changes, and the RCP checker only runs on closed submissions.

github-actions · 2026-08-31T09:20:59Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

ShriyaRishab · 2026-08-31T18:30:41Z

@chriscai-amd shouldn't open division have the same constraint as closed?
The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

Both divisions left opt_base_learning_rate unconstrained apart from positivity, so a submission could report any learning rate while still being scored against RCPs collected on a single recipe. The reference sweep follows a linear scaling law -- 1e-6 at global batch 8192, 2e-6 at 16384, 4e-6 at 32768, i.e. 1e-6 * GBS / 8192 -- and convergence is only comparable near it. Confine the learning rate to half to 1.5x that value, deriving the bounds in the global_batch_size POST block where the batch size is already parsed. Because the rules execute in log order, the checks guard on the bounds being resolved so a log that reports the learning rate before the batch size fails with the state dump rather than raising inside the comparison. The band applies to the open division too. That division is meant to showcase algorithmic changes, not to let submitters buy faster convergence with a learning rate the closed division would reject. Because the open rules do not pin the embedding rate to the dense one, opt_sparse_base_learning_rate is banded independently there; in the closed division the existing equality rule already covers it. Co-authored-by: Cursor <cursoragent@cursor.com>

chriscai-amd · 2026-08-31T18:44:39Z

@chriscai-amd shouldn't open division have the same constraint as closed? The open division is to showcase algorithmic changes but it shouldn't be a place where submitters just change the LR more than allowed by the rules to get faster convergence and better scores. That would not be in the spirit of the constraint.

thanks, @ShriyaRishab , updated the PR to restrict both the closed and open divisions

chriscai-amd marked this pull request as ready for review August 31, 2026 09:21

chriscai-amd requested review from a team as code owners August 31, 2026 09:21

chriscai-amd force-pushed the chcai/update_lr_range branch from cf19554 to 81b4883Compare August 31, 2026 18:42

pavanky approved these changes Sep 1, 2026
View reviewed changes