[v3] feat: concurrent test execution via `concurrency` by monikon22 · Pull Request #67 · Drownek/plugwright

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 · 2026-08-30T14:46:58Z

Closes#64.

test() and describe.serial() had no way to put several bots on the same feature at once — everything runs as one instance, sequentially, awaited one at a time. Races between real players (two bots claiming the same chest, buying the last item in stock) never got exercised.

`{ concurrency: N }`

test('only one player can claim the chest',{concurrency: 3},async({ player })=>{player.chat('/claim');awaitexpect(player).toHaveReceivedMessage(/Claimed|alreadyclaimed/);});

N independent instances run at once, each with its own bot leased from the pool. One instance failing fails the whole result — this is for finding a race, not averaging a pass rate. describe.serial blocks take the same option and run N full copies of the ordered chain.

Concurrency is validated against AccountPool.capacity() before any test in the session runs (every spec file is loaded up front for this — see runner.ts), so concurrency: 10 against a 4-account pool fails immediately instead of the 5th bot hanging on a lease. LocalMode has no pool and nothing to check; its ceiling is max-players.

Two bugs concurrency exposed

session.consoleLog.clear() at the top of every test would have raced two concurrent tests wiping each other's messages. Replaced with a non-destructive cursor: ServerWrapper captures session.consoleLog.length at construction, and toHaveReceivedMessage defaults to reading from there instead of index 0.
createBotScope.close() called session.disconnectAllBots() with no arguments, tearing down every bot in the session — harmless sequentially, but the first concurrent instance to finish would have kicked every still-running sibling. Fixed to keep every bot outside its own scope.

Report shape

Still one row per test/test-in-block — concurrency multiplies bots, not report rows. durationMs is the slowest instance; a new instances array carries every instance's own outcome (bot username, pass/fail, duration), so a failure names which bot lost. Wired into the console summary and the JSON report; JUnit keeps the single aggregate.

Console log labeling

Test:, Serial block:, PASSED/FAILED, and bot-creation lines get an [i/N] tag — N instances logging the same test name at the same time was unreadable without it:

Test: concurrent bots each see their own marker and stay connected [1/3]
[Bot] Creating bot: pw_9c27 [1/3]
Test: concurrent bots each see their own marker and stay connected [2/3]
[Bot] Creating bot: pw_dfc2 [2/3]
Test: concurrent bots each see their own marker and stay connected [3/3]
[Bot] Creating bot: pw_8c5d [3/3]
...
PASSED [1/3] (2.5s)
PASSED [2/3] (2.9s)
PASSED [3/3] (3.0s)

Docs

Writing Tests gets a new section on concurrency; Reports gets the aggregated JSON shape.

Checked

tsc --noEmit clean. plugwrightTestLocal: 54/54 pass, run three times across the changes.

session.consoleLog.clear() wiped the whole shared buffer at the start of every test. Fine when only one test runs at a time, but it means two tests running concurrently would race to wipe each other's messages out from under them. ServerWrapper now captures its own startIndex at construction (and can resetCursor() to "now"), and the toHaveReceivedMessage matcher defaults to reading from that instead of index 0 when since isn't given. Same observable behavior for a solo test, but the log itself is never destroyed, so nothing racing to read it can lose messages.

test(name, { concurrency: N }, fn) and describe.serial(name, { concurrency: N }, fn) fan out into N independent instances running at once, each with its own bot leased from the account pool. One failing instance fails the whole result — this is for races between real players, not a pass-rate to average. - Whole-session preflight: every spec file is loaded (imported once, registrations snapshotted) before any test runs, and every concurrency value is checked against the account pool's capacity() up front. A misconfigured concurrency aborts immediately instead of the Nth lease() hanging mid-run. An environment with no pool (LocalMode mints a throwaway account per bot) has nothing to check. - createBotScope.close() used to call session.disconnectAllBots() with no arguments, tearing down every bot in the session rather than just its own scope's. Harmless when nothing ran concurrently; with concurrency it would mean one finishing instance kicking every still-running sibling's bot. Fixed to keep every bot outside its own scope. - The report still has one row per test/test-in-block. Its durationMs is the slowest instance, and a new instances array carries every instance's own outcome (bot username, pass/fail, duration) so a failure names which bot lost the race. Wired into the console summary and the JSON report; JUnit keeps the single aggregated pass/fail, no per-instance breakdown. - A concurrent describe.serial block runs N full copies of the block; instances can diverge mid-block (one loses its race and stops early while another keeps going), so a test position only counts as skipped if every instance skipped it. - Console log lines (Test:, Serial block:, PASSED/FAILED, bot creation) are tagged with [i/N] — N instances logging the same test name at the same time is unreadable without it.

Covers both shapes: a plain test with concurrency, and a concurrent describe.serial block. Each instance checks its own marker against the shared server log and confirms it's still connected afterward — the two bugs concurrency exposed (destructive log clear, scope-wide bot teardown) would show up here as flaky or crashed instances.

Writing Tests gets a new section covering test()/describe.serial with concurrency: what it's for, what you get back, how many instances you can ask for, and why the server log stays shared and unfiltered across instances. Reports gets the aggregated JSON shape (instances array, botUsername) and a note that JUnit only ever sees the one aggregate.

…ut isn't full expect(server).toHaveReceivedMessage() needs consoleOutput: 'full'. The stand environment's RCON console only offers 'responses', same limitation simple-ts.spec.ts's 'server logs command execution' already declares — this test needed the same requires and didn't have it, so it failed test-example-plugin-stand in CI instead of skipping.

Two additions, both surfacing data the aggregate result already had: - TestInstanceResult gets index (1-based, matching the [i/N] console log tag for that same run) — was missing from both the console detail breakdown and the JSON report. - The summary table's own row for a concurrent test now tags itself with [passed/total] next to the duration, instead of that count only showing up in the Failed Tests detail section below. A passing concurrent test previously gave no indication in the table that it was even concurrent.

monikon22 added 6 commits August 30, 2026 17:37

Conversation

monikon22 commented Aug 30, 2026

{ concurrency: N }

Two bugs concurrency exposed

Report shape

Console log labeling

Docs

Checked

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

monikon22 commented Aug 30, 2026

{ concurrency: N }

Two bugs concurrency exposed

Report shape

Console log labeling

Docs

Checked

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

monikon22 commented Aug 30, 2026

{ concurrency: N }

Two bugs concurrency exposed

Report shape

Console log labeling

Docs

Checked

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

monikon22 commented Aug 30, 2026

{ concurrency: N }

Two bugs concurrency exposed

Report shape

Console log labeling

Docs

Checked

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

monikon22 commented Aug 30, 2026

{ concurrency: N }

Two bugs concurrency exposed

Report shape

Console log labeling

Docs

Checked

Uh oh!

Reviewers

Assignees

Labels

`{ concurrency: N }`

`{ concurrency: N }`

`{ concurrency: N }`

`{ concurrency: N }`

`{ concurrency: N }`

`{ concurrency: N }`

`{ concurrency: N }`