json.dumps() should encode float number NaN to null

[HaoyuSUN]

BPO	40633
Nosy	@rhettinger, @mdickinson, @ericvsmith, @alucab

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-05-18.08:08:12.857>
created_at = <Date 2020-05-15.13:42:04.329>
labels = ['type-feature', 'library', '3.9']
title = 'json.dumps() should encode float number NaN to null'
updated_at = <Date 2020-12-30.12:06:37.163>
user = 'https://bugs.python.org/HaoyuSUN'

bugs.python.org fields:

activity = <Date 2020-12-30.12:06:37.163>
actor = 'mark.dickinson'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-18.08:08:12.857>
closer = 'mark.dickinson'
components = ['Library (Lib)']
creation = <Date 2020-05-15.13:42:04.329>
creator = 'Haoyu SUN'
dependencies = []
files = []
hgrepos = []
issue_num = 40633
keywords = []
message_count = 21.0
messages = ['368942', '368944', '368948', '368950', '368959', '369005', '369006', '369043', '369044', '369172', '369188', '369189', '369190', '369191', '369194', '381481', '381491', '383986', '384055', '384059', '384060']
nosy_count = 6.0
nosy_names = ['rhettinger', 'mark.dickinson', 'eric.smith', 'Haoyu SUN', 'arjanstaring', 'alucab']
pr_nums = []
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40633'
versions = ['Python 3.9']

[HaoyuSUN]

Float numbers in Python can have 3 special number: nan, inf, -inf, which are encoded by json module as "NaN", "Infinity", "-Infinity". These representations are not compatible with JSON specifications RFC7159:
https://tools.ietf.org/html/rfc7159.html#page-6

These values are not correctly parsed by most JavaScript JSON encoders.

It is better to encode "NaN" to "null" which is a valid JSON keyword representing "Not a Number".

Here is an example how json.dumps() encodes NaN to NaN in JSON:
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> dct = {'a': None, 'b' : float('nan')}
>>> dct
{'a': None, 'b': nan}
>>> import json
>>> json.dumps(dct)
'{"a": null, "b": NaN}'

[ericvsmith]

Since this is documented behavior (https://docs.python.org/3.8/library/json.html#infinite-and-nan-number-values), we can't change it by default without breaking code.

What JavaScript JSON encoders and decoders specifically have a problem with this behavior? The documentation says "This behavior is not JSON specification compliant, but is consistent with most JavaScript based encoders and decoders.", so if there are encoders and decoders that it doesn't work with, that would be good to know.

[HaoyuSUN]

Thank you for the timely reply, Eric.

How about we add an optional argument (like the argument "ignore_nan" defaults to False as the package simplejson does) to functions like json.dumps(). So that user can choose whether he needs NaN encoded as NaN or null, meanwhile the default behavior stays the same.

In chromium based browsers, the function JSON.parse cannot parse it correctly. Here is an example below:

JSON.parse('{"a": null, "b": NaN}')
uncaught SyntaxError: Unexpected token N in JSON at position 17
at JSON.parse (<anonymous>)
at <anonymous>:1:6

[ericvsmith]

I think that's reasonable, although I could see someone objecting ("just use simplejson instead").

I suggest discussing this on the python-ideas mailing list and see what people think over there. It might help to create a PR first, if it's not a lot of work.

[mdickinson]

I don't think "null" in JSON is supposed to represent "Not a Number"; it's closer in meaning to Python's None. I definitely wouldn't want to see nans translated to "null" by default.

This also only seems to address a part of the issue: what's the proposed action for "Infinity" and "-Infinity"? We've written internal code to deal with float special values in JSON a few times (usually to work with databases that stick to the strict JSON definition), and that code has to find a way to deal with all three of the special values.