Simulation state is process-wide, not per-simulation

[HugoFara]

ForeFire keeps simulation state in process-wide storage — statics, class-wide members, and one singleton — rather than in the objects that own a simulation. A FireDomain is not a self-contained simulation; it is one of several users of a shared pool.

Under a normal CPython this is invisible: the binding never releases the GIL, so every entry point is serialised. Take the GIL away and eight concurrent simulations crash or return wrong answers, measured below.

What is shared

All line numbers against dev at e1df887.

State	Where	Scope today
SimulationParameters::instance	SimulationParameters.cpp:29	One parameter set for the whole process, whatever GetInstance() is called on.
ForeFireAtom::instanceNRCount	ForeFireAtom.h:106	Every atom's id comes from one counter, incremented with a plain ++.
FireDomain::propModelsTable, fluxModelsTable	FireDomain.cpp:58-59	Fixed arrays of 50 and 500 slots, scanned for a free index at FireDomain.cpp:859 and written at 697. Two domains compete for the same slots.
FireNode::nmlScheme, smoothing, relax, minSpeed, minFrontDepth	FireNode.cpp:22-30	Numerical settings held per class, so every node in the process shares one value.
StringRepresentation::outputstr	StringRepresentation.cpp:29	One ostringstream shared by every representation object.

There are no concurrency primitives anywhere in src/ except the HTTP server's running flag: searching roughly 33,000 lines for std::mutex, std::atomic, lock_guard, pthread_mutex or OpenMP directives returns that single match.

Measured

Eight threads, each building a 1000×1000 domain with the Iso propagation model, igniting at the centre and stepping five times. Each thread must reproduce the fire node count it produces when run alone. CPython 3.14.6 free-threading, manylinux_2_28 container, wheel built from source.

Build	Same 8 threads, GIL on	GIL off, 5 runs
dev @ e1df887	3/3 pass	4 × segfault, 1 × wrong results
dev + the four small fixes below	—	4 × segfault, 1 × abort
full branch (all 8 commits)	pass	5/5 pass

The GIL-on column is the control: identical workload, identical code, and it passes every time. The only variable is whether the calls are serialised.

The one GIL-off run on dev that did not crash returned this:

single-threaded baseline: 62 fire nodes
FAILED with 7 problem(s):
 - thread 1 produced 49 fire nodes, expected 62
 - thread 2 produced 46 fire nodes, expected 62
 - thread 3 produced 209 fire nodes, expected 62
 - thread 4 produced 27 fire nodes, expected 62

Silently wrong output is worse than the segfaults. The single-threaded baseline is 62 nodes on both dev and the fixed branch, so the change below does not alter results.

The four small fixes do not fix concurrency on their own, as the middle row shows. That is not an argument against them — they are each a correctness fix — but nobody should read them as making threaded use safe. Safety arrives only with the state refactor in step 5.

Why it matters, in two directions

More than one simulation per process. Command.cpp:187 already carries a special case for a second FireDomain[...], keyed on the domain ID being 1, which is the shape a shared model table forces. The ensembles work in #172 wants many simulations, and today each one has to be its own process. Note this consequence is read from the code, not measured — the crashes above are the threading half.

Free-threaded Python. #155 stopped shipping cp314t wheels because of the table above, and closed with "worth its own issue if anyone wants it". This is that issue, and the measurements say the call in #155 was right: the GIL is what currently makes ForeFire safe.

Proposed order

Work is drafted on a branch and splits into parts with very different risk. I would like the first four to land independently, each defensible on its own terms whether or not the larger change happens.

1. The concurrency stress test (tests/python/test_threading.py, new file) — the harness that produced the table above. Skips loudly on an interpreter with the GIL on, so it cannot pass vacuously. No core changes.
2. Atomic id counter and a safe singleton. instanceNRCount becomes std::atomic<long> with a relaxed fetch_add; GetInstance becomes a function-local static, whose once-only initialisation C++11 already guarantees. Eight lines, no single-threaded behaviour change.
3. A per-instance output buffer for StringRepresentation, replacing the shared static.
4. Serialise entry into NetCDF and HDF5. The NetCDF C library is not thread-safe unless built for it; the lock is additive and uncontended in a single-threaded run.

Then the substantial part, which is a real API change and should not proceed without maintainer agreement:

5. Give each simulation its own state — move the model tables, the FireNode settings and the parameters off process-wide storage and onto the domain, and make Command and the Python binding instance-based. This touches the signatures of FireDomain, DataBroker, Command and CLibForeFire, so it needs a decision about the coupling ABI before it is worth polishing.
6. Declare py::mod_gil_not_used() and resume cp314t wheels — only once 5 is done, since it is the promise the rest makes true. This reverses part of Stop shipping free-threaded (cp314t) wheels #155 deliberately.

Steps 1–4 rebase cleanly onto dev today. Step 5 conflicts with DataBroker.cpp and Command.cpp and is still rough.

What would help

Whether multiple simulations in one process is something the project wants at all. If not, steps 1–4 are still worth having as plain correctness fixes, and 5–6 should be closed rather than left open. If yes, the coupling ABI is the thing to settle first, since CLibForeFire's extern "C" surface is what Meso-NH binds to.

---

Drafted by Claude Opus 5 from a codebase audit. Reviewed by a maintainer before filing.

[HugoFara]

One clarification on the StringRepresentation::outputstr row, after chasing it further: the shared buffer is safe under sequential use, and nothing in the table above should be read as a live single-threaded bug.

dumpStringRepresentation() resets the buffer with outputstr.str("") at StringRepresentation.cpp:341 before writing, and every write to it sits inside a visit() method reachable only through the domain->accept(this) on the next line. There is no other entry point.

That matters for the coupled path specifically. Command.cpp:205 and 232 create two representations in one session — outStrRepp and outStrRep — and both are inserted into the timetable as events, so both call output() during a run. Because each dump resets the buffer first, their outputs cannot mix. Only overlapping dumps can corrupt each other, which is the threading case already measured above.

The same applies to currentLevel (StringRepresentation.cpp:28), reset in the same place.

So step 3 in the plan is a design fix — a static buffer on a class that is instantiated more than once — rather than a repair of anything that misbehaves today. Recording it here so nobody goes looking for a coupled-run reproduction that does not exist.