Unified hash for numeric types.

[mdickinson]

BPO	8188
Nosy	@rhettinger, @mdickinson, @ericvsmith, @skrah
Files	numeric_hash.patch numeric_hash2.patch numeric_hash3.patch numeric_hash4.patch numeric_hash5.patch numeric_hash6.patch doc_stdtypes.patch: apply after numeric_hash6.patch numeric_hash7.patch numeric_hash8.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/mdickinson'
closed_at = <Date 2010-05-23.13:35:21.075>
created_at = <Date 2010-03-20.20:34:28.514>
labels = ['type-feature']
title = 'Unified hash for numeric types.'
updated_at = <Date 2010-06-11.10:51:51.694>
user = 'https://github.com/mdickinson'

bugs.python.org fields:

activity = <Date 2010-06-11.10:51:51.694>
actor = 'mark.dickinson'
assignee = 'mark.dickinson'
closed = True
closed_date = <Date 2010-05-23.13:35:21.075>
closer = 'mark.dickinson'
components = []
creation = <Date 2010-03-20.20:34:28.514>
creator = 'mark.dickinson'
dependencies = []
files = ['16604', '16608', '16610', '16629', '16675', '16756', '16782', '16907', '17437']
hgrepos = []
issue_num = 8188
keywords = ['patch']
message_count = 25.0
messages = ['101395', '101397', '101401', '101402', '101403', '101404', '101405', '101406', '101407', '101417', '101581', '101824', '102339', '102341', '102347', '102350', '102352', '102467', '103031', '103032', '103800', '103843', '106291', '106333', '107539']
nosy_count = 6.0
nosy_names = ['rhettinger', 'mark.dickinson', 'Rhamphoryncus', 'casevh', 'eric.smith', 'skrah']
pr_nums = []
priority = 'normal'
resolution = 'accepted'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue8188'
versions = ['Python 3.2']

[mdickinson]

Here's a patch that makes hash(x) == hash(y) for any numeric types (int, float, complex, Decimal, Fraction, bool) when x and y are numerically equal.

This is a prerequisite for making all numeric types accurately comparable with each other.

[mdickinson]

Uploaded to Rietveld:

http://codereview.appspot.com/660042

[mdickinson]

Updated patch, with a bit of cleanup and some comments describing the hashing strategy; I'll update the Rietveld issue as well.

[mdickinson]

Whoops; that patch included some accidental Lib/test/test_decimal changes. Here's the correct patch.

[Rhamphoryncus]

Why aren't you using 64-bit hashes on 64-bit architectures?

[mdickinson]

Why aren't you using 64-bit hashes on 64-bit architectures?

Mostly because I haven't got around to putting that in yet. :)

Ideal would be to use _PyHASH_BITS=61 for 64-bit machines, throughout.

[Rhamphoryncus]

I assume you mean 63. ;)

[mdickinson]

No, I mean 61. 2**61 - 1 is prime; 2**63-1 is not. (So 2 bits of the hash get wasted, but that's not a big deal, especially since they're the high-end bits and Python mostly cares about the lower-order bits.)

[mdickinson]

Restore tests accidentally omitted from second patch.

[mdickinson]

Updated patch:

• put hash parameters into pyport.h, to avoid repetition; make them
  available to Python code via a private attribute sys._hash_info.

• use a modulus of 2**61-1 on systems where SIZEOF_LONG >= 8, and
  a modulus of 2**31 - 1 otherwise.

• remove _invmod from fractions module. It's faster (and easier) to
  use 3-argument pow to compute inverses modulo a prime.

• add a few more tests.

[mdickinson]

Another update, partly to address comments raised by Guido on Rietveld. I'll upload these changes to Rietveld later today.

• rename sys._hash_info to sys.hash_info and make it public rather than private (it still needs docs somewhere)

• add some explanatory comments to long_hash; remove an outdated comment

• fix missing error check (in previous patch) in slot_tp_hash. slot_tp_hash also now always raises a TypeError if __hash__ returns a non-integer; this is a change from current behaviour, which allows small floats to be returned by __hash__, but not large floats (where large means > 2**31 or > 2**63 in absolute value, depending on the system). I'm assuming this was unintentional (the docs specify that __hash__ should return an integer).

• simplify specification of hash function slightly: for nonnegative x it simply computes the reduction of x; previously it computed 1 + reduction of (x-1) for positive values. This extra +-1 doesn't really add anything of value, and makes it slightly more complicated and error-prone to write your own hash function.

[mdickinson]

Here's a version of the patch that adds exact comparisons between the various numeric types. The only slightly tricky comparison is the Fraction <-> Decimal one: an obvious strategy is to convert the Decimal exactly to a Fraction and then use the fraction comparison, but this is inefficient for Decimal instances with large exponent. So instead, we compare a Decimal x with a Fraction n/d by comparing x*d with n in the Decimal domain.

[mdickinson]

New patch:

• document and test sys.hash_info
• document numeric hash definition (in Doc/library/stdtypes.rst; I'm
  not sure whether this is the best place for it)
• document Decimal change (Decimal instances are now comparable
  with instances of float, fraction.Fraction)
• refresh patch to apply cleanly to current svn.

I think this is close to final form: I intend to apply this patch (or something very much like it) soon; any review would be appreciated.

[mdickinson]

I've refreshed the Rietveld patch as well:

http://codereview.appspot.com/660042