pickle does not serialize Exception __cause__ field

[diekhans]

BPO	29466
Nosy	@brettcannon, @ncoghlan, @diekhans, @avassalotti, @ericsnowcurrently, @serhiy-storchaka, @tjb900, @kakshay21, @iritkatriel
Files	cause_pickle.py: bug demog program

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-02-06.18:36:36.651>
labels = ['type-bug', 'library', '3.11']
title = 'pickle does not serialize Exception __cause__ field'
updated_at = <Date 2022-01-06.16:21:58.240>
user = 'https://github.com/diekhans'

bugs.python.org fields:

activity = <Date 2022-01-06.16:21:58.240>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-02-06.18:36:36.651>
creator = 'diekhans'
dependencies = []
files = ['46550']
hgrepos = []
issue_num = 29466
keywords = []
message_count = 6.0
messages = ['287163', '287304', '335686', '396603', '396604', '409871']
nosy_count = 9.0
nosy_names = ['brett.cannon', 'ncoghlan', 'diekhans', 'alexandre.vassalotti', 'eric.snow', 'serhiy.storchaka', 'tjb900', 'kakshay', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29466'
versions = ['Python 3.11']

[diekhans]

python3 pickle does not serialize the __cause__ field, as shown by the attached demo program.

[serhiy-storchaka]

True. Attributes __context__, __cause__ and __traceback__ are not pickled. The traceback objects are even not pickleable.

What is worse, some other non-special attributes are lost during pickling. For example name and path attributes of ImportError.

>>> try: import foo
... except Exception as ex: exc = ex
... 
>>> exc.name
'foo'
>>> exc.__reduce__()
(<class 'ModuleNotFoundError'>, ("No module named 'foo'",), {})

Or the value attribute of StopIteration if it was not passed to the constructor.

>>> exc = StopIteration()
>>> exc.value = 42
>>> exc.__reduce__()
(<class 'StopIteration'>, (), {})

[kakshay21]

Hey, can I work on this?

[iritkatriel]

I get different output for Serhiy's first example now, but the same for the second:

>>> try: import foo
... except Exception as ex: exc = ex
... 
>>> exc.name
'foo'
>>> exc.__reduce__()
(<class 'ModuleNotFoundError'>, ("No module named 'foo'",), {'name': 'foo'})
>>> exc = StopIteration()
>>> exc.value = 42
>>> exc.__reduce__()
(<class 'StopIteration'>, ())
>>>

[iritkatriel]

See also bpo-43460, bpo-32696, bpo-30005.

[iritkatriel]

Attributes __context__, __cause__ and __traceback__ are not pickled. The traceback objects are even not pickleable.

It's not guaranteed that the __cause__ and __context__ are picklable either.

Should we try to pickle them, or should we document this and suggest traceback.TracebackException for storing or transmitting exception information?

[csm10495]

Some stuff may be lost in time here, though for my benefit: Why don't the fields on exceptions go into __dict__? If they were there wouldn't serialization work closer-to 'for free'?