By default when creating subprocesses, windows and macOS “spawn” new processes, while on Linux you “fork”. https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
Spawn is slow, but makes a new clean python instance with nothing preloaded. Fork is faster, and also you inherit whatever you had in Python before. For ActivitySim, in the MP init step, the subprocess may have access to previously loaded tables, but not the random number generator. Since the ‘init’ subprocess sees the tables that are there, it doesn’t reload them, and doesn’t trigger the random value generator initialization. If the init subprocess then calls for a random number, it will crash.
Possible solutions include using spawn on all platforms for consistency, or checking and/or re-initializing random number generators, tracing setup, etc, upon opening a pipeline, or force-wiping existing tables and reloading them when opening a pipeline.
By default when creating subprocesses, windows and macOS “spawn” new processes, while on Linux you “fork”. https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
Spawn is slow, but makes a new clean python instance with nothing preloaded. Fork is faster, and also you inherit whatever you had in Python before. For ActivitySim, in the MP init step, the subprocess may have access to previously loaded tables, but not the random number generator. Since the ‘init’ subprocess sees the tables that are there, it doesn’t reload them, and doesn’t trigger the random value generator initialization. If the
initsubprocess then calls for a random number, it will crash.Possible solutions include using spawn on all platforms for consistency, or checking and/or re-initializing random number generators, tracing setup, etc, upon opening a pipeline, or force-wiping existing tables and reloading them when opening a pipeline.