Run time-based snapshot tests concurrently

[Amaury Chamayou (achamayou)]

Summary

• run the low, exact, and high time-based snapshot scenarios concurrently
• preserve separate network configurations and propagate worker failures
• overlap their fixed observation windows and their network setup/teardown cycles

Measured effect

The affected code runs inside schema_test (tests/schema.py -> e2e_operations.run -> run_time_based_snapshotting), the longest test in bucket_b. ctest runs bucket_b serially, so the saving passes straight through to the job.

Comparing 2 CI runs on this branch against 8 recent main runs:

Measure	main (n=8)	This PR (n=2)	Delta
schema_test	419.2s	371.3s	-47.9s (-11.4%)
schema_test, normalised (see below)	419.2s	368.4s	-50.8s (-12.1%)
bucket_b ctest total	844.3s	799.8s	-44.5s (-5.3%)
VMSS Virtual B wall clock	1172.5s	1139.5s	-33s (noisy)

All figures are means. Individual schema_test samples: main 413.7 / 413.7 / 414.0 / 414.1 / 416.5 / 418.8 / 428.4 / 434.7; this PR 364.7 / 378.0.

Normalisation. Runner speed varies noticeably between runs, so the five unchanged tests in bucket_b (recovery_test, recovery_stale_snapshot_join_test, recovery_intermediate_snapshot_join_test, recovery_snapshot_endorsements_test, nodes_test) are used as a control. Their total was 425.1s on main versus 428.4s here, i.e. the runners for this branch were ~0.8% slower, so the raw delta slightly understates the saving.

Two notes:

• This beats the ~20s originally estimated from the fixed 10s observation windows alone, because parallelising also overlaps three network start/shutdown cycles.
• VMSS Virtual B is the critical path of the CI run (VMSS Virtual C ~1050s, VMSS Virtual A ~800s), so this comes off end-to-end CI time. B remains critical afterwards, but the margin over C narrows from ~100s to ~50s. Job wall clock is too noisy to confirm the saving on its own; the ctest figures are the reliable measure.

[Copilot]

Pull request overview

This PR parallelises the three time-based snapshotting scenarios inside run_time_based_snapshotting() to reduce the wall-clock time of the schema_test path in bucket_b, while keeping each scenario’s network configuration isolated via per-scenario args.label and snapshot_min_tx_interval.

Changes:

• Run the low/exact/high time-based snapshot scenarios concurrently using concurrent.futures.ThreadPoolExecutor.
• Refactor each scenario into its own run_* worker function to preserve separate network setup/teardown per scenario.

Custom instructions used:

• None (no .github/copilot-instructions.md or .github/instructions/* files were loaded during this review)

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.