Issues can arise when clients and workers don't agree on versions of libraries like cloudpickle . This is a common user issue.
We often recommend calling client.get_versions(check=True) which errs in a very informative way when versions do not match. We have talked about running this automatically on startup but have avoided it so far.
Perhaps we should run this, but just warn instead of err. I propose the following:
- Support warning rather than erring in get_versions with
get_versions(check='warn'). Valid arguments to this would be 'warn', 'err', 'error', and True the last three of which would all alias to the same behavior.
- Call get_versions in
client._start
- Remove this functionality when automatically starting clients, such as in
worker.py::get_client() or Future.__setstate__
Issues can arise when clients and workers don't agree on versions of libraries like cloudpickle . This is a common user issue.
We often recommend calling
client.get_versions(check=True)which errs in a very informative way when versions do not match. We have talked about running this automatically on startup but have avoided it so far.Perhaps we should run this, but just warn instead of err. I propose the following:
get_versions(check='warn'). Valid arguments to this would be'warn', 'err', 'error', and Truethe last three of which would all alias to the same behavior.client._startworker.py::get_client()orFuture.__setstate__