Skip to content

BUG: inf in quantile has undefined behaviour (and possibly different for -inf vs +inf) #21091

Description

@jessexknight

Describe the issue:

  1. When one or more inf or -inf are present in the argument to np.quantile (or np.nanquantile), the results often include nan, when +/-inf could be reasonably returned -- e.g. if there are 10 -infs in 100-long x, np.quantile(x,.05) should probably return -inf, not nan.

  2. np.quantile(-inf,0) =/= np.quantile(-inf,0.) (int vs float, and similarly for 1 vs 1.)

  3. The behaviour for -inf and inf is possibly different in some situations -- e.g. compare actual outputs 7 vs 9 below: the median in 7 that averages 2 and inf returns nan while the median in 9 that averages -inf and 3 returns -inf.

Likely related: #12282

Reproduce the code example:

import numpy as np
inf = np.inf
nan = np.nan
eps = 1e-9
x_pos_even = [1,2,inf,inf]
x_pos_odd  = [1,2,3,inf,inf]
x_neg_even = [-inf,-inf,3,4]
x_neg_odd  = [-inf,-inf,3,4,5]
q_even     = [0,1/3,2/3,1]
q_odd      = [0,.25,.5,.75,1]
printfun = lambda r: print(np.round(r,2))
printfun(np.quantile(x_neg_even,0))
printfun(np.quantile(x_neg_even,0.))
printfun(np.quantile(x_pos_even,q_even))
printfun(np.quantile(x_pos_odd, q_even))
printfun(np.quantile(x_neg_even,q_even))
printfun(np.quantile(x_neg_odd, q_even))
printfun(np.quantile(x_pos_even,q_odd))
printfun(np.quantile(x_pos_odd, q_odd))
printfun(np.quantile(x_neg_even,q_odd))
printfun(np.quantile(x_neg_odd, q_odd))

# expected output -- nan* = truly undefined behaviour, though could possibly be +/- inf
# -inf
# -inf
# [ 1.  , 2.  , inf , inf ]
# [ 1.  , 2.33, nan*, inf ]
# [-inf ,-inf , 3.  , 4.  ]
# [-inf , nan*, 3.67, 5.  ]
# [ 1.  , 1.75, nan*, inf , inf ]
# [ 1.  , 2.  , 3.  , inf , inf ]
# [-inf ,-inf , nan*, 3.25, 4.  ]
# [-inf ,-inf , 3.  , 4.  , 5.  ]

# actual output (spacing adjusted for readability)
# -inf
# nan
# [ 1.  , nan , nan , nan ]
# [ 1.  , 2.33, nan , nan ]
# [ nan , nan , 3.  , 4.  ]
# [ nan , nan , 3.67, 5.  ]
# [ 1.  , 1.75, nan , nan , nan ]
# [ 1.  , 2.  , nan , nan , nan ]
# [ nan , nan ,-inf , 3.25, 4.  ]
# [ nan , nan , 3.  , 4.  , 5.  ]

Error message:

/usr/local/lib/python3.8/dist-packages/numpy/lib/function_base.py:4486: RuntimeWarning: invalid value encountered in subtract
  diff_b_a = subtract(b, a)
/usr/local/lib/python3.8/dist-packages/numpy/lib/function_base.py:4488: RuntimeWarning: invalid value encountered in multiply
  lerp_interpolation = asanyarray(add(a, diff_b_a * t, out=out))
/usr/local/lib/python3.8/dist-packages/numpy/lib/function_base.py:4489: RuntimeWarning: invalid value encountered in subtract
  subtract(b, diff_b_a * (1 - t), out=lerp_interpolation, where=t >= 0.5)
/usr/local/lib/python3.8/dist-packages/numpy/lib/function_base.py:4488: RuntimeWarning: invalid value encountered in add
  lerp_interpolation = asanyarray(add(a, diff_b_a * t, out=out))

NumPy/Python version information:

1.22.2 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions