Make Custom Object Classes JSON Serializable

[andrewchap]

BPO	35111
Nosy	@rhettinger, @etrepum

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/etrepum'
closed_at = None
created_at = <Date 2018-10-30.11:43:02.520>
labels = ['type-feature', '3.8']
title = 'Make Custom Object Classes JSON Serializable'
updated_at = <Date 2018-10-31.18:19:47.348>
user = 'https://bugs.python.org/andrewchap'

bugs.python.org fields:

activity = <Date 2018-10-31.18:19:47.348>
actor = 'bob.ippolito'
assignee = 'bob.ippolito'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2018-10-30.11:43:02.520>
creator = 'andrewchap'
dependencies = []
files = []
hgrepos = []
issue_num = 35111
keywords = []
message_count = 5.0
messages = ['328897', '328968', '328971', '329003', '329004']
nosy_count = 3.0
nosy_names = ['rhettinger', 'bob.ippolito', 'andrewchap']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue35111'
versions = ['Python 3.8']

[andrewchap]

When creating a custom class that doesn't inherit from the accepted classes, there is no way to serialize it using json.dumps or json.dump.

I propose that __fromjson__ and __tojson__ that when present will be used by the Python's default encoder. This issue is widely documented on stackoverflow as a deficiency.

You basically have a couple of options to solve this:

1. Implement a custom json.JSONEncoder/json.JSONDecoder
2. Implement a method like to/from json
3. Monkey patch the json library
4. Use a 3rd party library

These are not very good options. If you can serialize an object using pickle, why not have the ability to serialize objects using json?

Thank you

[rhettinger]

If you can serialize an object using pickle,
why not have the ability to serialize objects using json?

One reason would be that the JSON spec was intentionally designed to handle a limited number of types so that it would have maximum interoperability between languages. Another reason is that in order to make other types round-trip between encoding and decoding, you would need to control both ends, in which case, you might as well use pickle.

[etrepum]

The trouble with having such a hook is that it would take precedence over any customization you might want or need to do to satisfy the protocol you're implementing. Other than the limited set of types that are part of the JSON specification, there's essentially no standard for encoding of anything else. This is why customization is left to the call sites for encoding and decoding, and I would recommend using the default and object_pairs_hook keyword arguments to dumps and loads respectively for that, rather than any of the options that you've enumerated.

For what it's worth, simplejson has had a for_json method hook for several years to encourage some consolidation, but it's opt-in and there hasn't been a lot of demand to make it the default. The inverse from_json type operation is not supported. If you think about it, how could you even implement such a thing in the general case, in a way that wouldn't have lots of surprises and performance issues?

[andrewchap]

Bob,

I understand what you are saying, but having a built-in is way easier than encoder/decoders. You can still have both actually. Even if you didn't have two way, a one way would be amazingly helpful. For large development systems, a __json__ method would make serialization super easy.

class foo(object):
 a = 1
 b = [1,2,3]
 def __json__(self): return {'a':self.a,'b':self.b}

[etrepum]

That's what the for_json method is in simplejson, it does not have widespread usage.

You can implement that when encoding:

def json_default(obj):
 try:
 return obj.__json__()
 except AttributeError:
 raise TypeError("{} can not be JSON encoded".format(type(obj)))


json.dumps(foo(), default=json_default)

This way, you can choose precisely how the output needs to work when encoding. It's not ideal for every use case, but nothing is. The point is that it doesn't get in your way, whatever you need to do can be done without any awful tricks, so long as you have control over the dumps call site.

[dennisvang]

Similar to #71549?

[dennisvang]

... This issue is widely documented on stackoverflow ....

Just some references to illustrate this point:

• How to make a class JSON serializable (>1.4 million views)
• Making object JSON serializable with regular encoder (>75k views)