References to global objects are broken

[cicdw]

In cloudpickle v.0.8.0 and above, it seems that references to global objects are lost in the serialization -> deserialization process.

For example, the following asserts pass in cloudpickle 0.7.0:

import cloudpickle


FOO = {}

def foo_add(x):
 FOO["x"] = x

func = cloudpickle.loads(cloudpickle.dumps(foo_add))

assert FOO == {}, FOO

func(66)
assert FOO == {"x": 66}, FOO

foo_add(67)
assert FOO == {"x": 67}, FOO

However, in 0.8.0 and above, the following assert fails:

func(66)
assert FOO == {"x": 66}, FOO

with:

AssertionError: {}

i.e., the function runs without error but doesn't actually affect the global dictionary anymore.

[pierreglaser]

Indeed, from 0.5.4 to 0.7.0, we experimented with maintaining state inside workers by making depickled functions look for existing globals inside their new environment. But maintaining stateful workers led to unexpected behaviors and complex bugs for users/developers of some downstream projects of cloudpickle such as joblib and ray (see #240), and in 0.8.0 we decided to restore cloudpickle's "old" behavior ( prior to 0.5.3), i.e not to use pre-existing globals into newly depickled functions. We actually now test the opposite of your bug report, i.e that newly-depickled functions do not reuse any pre-existing state from the worker they were depickled in.

If you want to maintain a state across depickled function calls, you can still do it by making FOO an attribute of some module. This script for instance:

In [1]: import cloudpickle
 ...: cloudpickle.FOO = {}
 ...:
 ...: def foo_add(x):
 ...: cloudpickle.FOO["x"] = x
 ...:
 ...: func = cloudpickle.loads(cloudpickle.dumps(foo_add))
 ...:
 ...: assert cloudpickle.FOO == {}, cloudpickle.FOO
 ...:
 ...: func(66)
 ...: assert cloudpickle.FOO == {"x": 66}, cloudpickle.FOO
 ...:
 ...: foo_add(67)
 ...: assert cloudpickle.FOO == {"x": 67}, cloudpickle.FOO

passes on my local machine.

By the way, we continuously build and run the test suite of some downstream projects relying on cloudpickle against cloudpickle master to make sure we do not break them. Was your report triggered by a bug on prefect? If you are interested, we can integrate prefect into our list of downstream projects to test - this would allow to tackle such situations earlier in our development process.

[cicdw]

Ah awesome, thanks for the detailed response @pierreglaser ! That makes sense.

It was triggered by how we had written some tests to track state, but not by any functionality in our code base. Ultimately prefect relies on cloudpickle because of dask, so our tests wouldn't catch anything that dask itself wouldn't catch. I appreciate the offer though!

Closing as this isn't a real bug.