Skip to content

flake: TestRun (natsbench multiple variants) #1612

Description

@flake-investigator

CI Failure Details

CI Run:https://github.com/coder/coder/actions/runs/28496003256
Failed Job:https://github.com/coder/coder/actions/runs/28496003256/job/84462298128 (test-go-pg (windows-2022))
Failure Time: 2026-07-01T05:52:31Z

Commit Info

Failing Tests

  • TestRunSingleNode
  • TestRunCluster

File: coderd/x/nats/natsbench/natsbench_internal_test.go (around lines ~43-105)

Error Output

=== FAIL: coderd/x/nats/natsbench TestRunSingleNode (0.03s)
natsbench_internal_test.go:69:
Error: "0" is not greater than "0"
=== FAIL: coderd/x/nats/natsbench TestRunCluster (1.26s)
natsbench_internal_test.go:104:
Error: "0" is not greater than "0"

Error Analysis

Both tests assert non-zero throughput (res.PubsPerSec / res.DeliveriesPerSec) after running the benchmark. The results reported 0, causing require.Greater(..., 0.0) to fail for both single-node and cluster runs.

Root Cause Assessment

Likely flaky timing/benchmark instability on Windows runners where the benchmark reports zero throughput despite successful setup. No evidence of infra-level failure beyond the zero metrics.

Data Race / Panic / OOM Checks

Searched the job logs for WARNING: DATA RACE, panic:, out of memory, signal: killed — none found.

Assignment Analysis

Intended blame targets:

  • git blame -L 43,72 coderd/x/nats/natsbench/natsbench_internal_test.go
  • git blame -L 74,104 coderd/x/nats/natsbench/natsbench_internal_test.go

File history:

  • git log --oneline -10 --follow coderd/x/nats/natsbench/natsbench_internal_test.go
    • Most recent substantive change: ea1379d42 ("chore: add nats benchmarking pkg") by Jon Ayers (PR #26396, authored by @sreya).

Assigning to @sreya as the current owner of the natsbench test area (original PR author and related NATS test owner).

Related Issues Search

Searched coder/internal for:

  • TestRunSingleNode
  • TestRunCluster
  • natsbench_internal_test.go
  • natsbench

No matching issues found.

Reproduction

go test ./coderd/x/nats/natsbench -run TestRun -count=1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions