Filter out SPMI metric diffs smaller than ±0.001%

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757

[Copilot]

Description

SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The write_metricdiff_markdown_summary function used a bare != equality check, so any tiny numeric variance was reported as significant.

Inlined an abs(compute_pct(...)) >= 0.01 threshold check directly in the list comprehension filter in superpmi.py to suppress metric diffs below ±0.01%. This is consistent with the existing is_significant_pct threshold pattern already used in write_tpdiff_markdown_summary.

A dummy comment change was also added to src/coreclr/jit/compiler.cpp to trigger the SPMI pipeline and validate the fix.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>JIT: Metrics show changes even with no-diff changes</issue_title>
<issue_description>E.g. https://dev.azure.com/dnceng-public/public/_build/results?buildId=1339686&view=ms.vss-build-web.run-extensions-tab has no diffs, yet several metrics show up as changing. We probably should increase the expected variance of these metrics so that they don't show up unless they are actually changing.

cc @EgorBo </issue_description>

<agent_instructions>Please do not report diffs for SPMI metric diffs if the difference is smaller than +/- 0.01%</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• FixesJIT: Metrics show changes even with no-diff changes #125683

---

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[EgorBo]

@copilot please make a dummy change in jit to trigger the spmi pipeline

[Copilot]

@copilot please make a dummy change in jit to trigger the spmi pipeline

Done in 97e1984 — added a comment line to compiler.cpp to trigger the SPMI pipeline.

[Copilot]

Pull request overview

This PR aims to reduce noise in SuperPMI “metricdiff” markdown reporting by suppressing insignificant metric changes that are due to tiny numeric variance rather than real codegen differences.

Changes:

• Updated write_metricdiff_markdown_summary to filter metric diffs using a percent-change threshold instead of a raw != comparison.

[EgorBo]

@jakobbotsch PTAL. copilot has an idea how to fix the variance - it blames SPMI itself, the fact that it re-uses CompilationResult, but I don't like its fix yet, so I think having a threshold won't hurt.

[EgorBo]

/ba-g deadletter + #125757