Fix decimal by srittau · Pull Request #2323 · python/typeshed

srittau · 2018-07-11T18:05:27Z

Fix various problems with decimal and add missing annotations.

srittau · 2018-07-12T08:30:16Z

mypy crash reported as python/mypy#5347.

srittau · 2018-07-13T10:38:42Z

Crash is now fixed in mypy master.

JelleZijlstra · 2018-07-13T15:04:07Z

Thanks! I restarted Travis, will take another look after it passes.

JelleZijlstra

Thanks! Left a few comments.

JelleZijlstra · 2018-07-14T02:26:53Z

-_ComparableNum = Union[Decimal, int, float]
+_DecimalNew = Union[Decimal, int, float, Text, Tuple[int, Sequence[int], int]]
+if sys.version_info >= (3,):
+ _ComparableNum = Union[Decimal, numbers.Rational, float]


This conditional definition makes it look confusingly like int is not supported in Python 3. In fact, it is redundant in Python 2 too, because it's automatically promoted to float. It would be clearer to have int either in both or in neither of the branches.

JelleZijlstra · 2018-07-14T02:37:04Z

- def __delattr__(self, name): ...
- def __reduce__(self): ...
- def clear_flags(self): ...
+ def __setattr__(self, name: str, value: Any) -> Any: ...


I think this should actually be omitted because the effect of having it is that the type checker will think setting any attribute is allowed. However, the implementation actually only allows the attributes that are known to exist on the class.

I replaced it with a comment saying so, for future reference.

JelleZijlstra · 2018-07-14T02:37:59Z

+ def is_canonical(self, a: _Decimal) -> Decimal: ...
+ def is_finite(self, a: _Decimal) -> Decimal: ...
+ def is_infinite(self, a: _Decimal) -> Decimal: ...
+ def is_nan(self, a: _Decimal) -> Decimal: ...


Most of these return bool, although I haven't checked all of those. I'm guessing divmod returns a Tuple[Decimal, Decimal].

Good catch. I am rechecking the return values of all Context methods.

srittau · 2018-07-14T09:25:12Z

I am not sure why the travis build does not start automatically after the latest patches.

JelleZijlstra · 2018-07-14T17:56:47Z

I closed and reopened, and now Travis seems to be running.

gvanrossum · 2018-07-26T04:30:15Z

I wonder if this broke something. I've got code that's essentially (though not quite)

float(Decimal() if bool() else 0)

(Though that example doesn't provoke it.)

The if/else expression's type is revealed as object, and
float(object()) is invalid. Maybe it was valid before? Or maybe the
join previously returned Union[Decimal, int] and now deteriorated to
object?

I kind of suspect #2323, since
it removed the SupportsFloat and SupportsInt base classes from Decimal
(the last typeshed sync before this afternoon's was
python/mypy#5373, which synced typeshed 3
commits before 2323).

srittau · 2018-07-26T08:31:04Z

I think it's reasonable to assume that this change triggered the problem, although I suspect a bug in mypy. Since SupportsAbs and SupportsFloat are protocols, they should not be needed as base classes.

The following test cast demonstrates the problem:

fromtypingimportSupportsFloatclassFloaty:
def__float__(self): ...
classSupporter(SupportsFloat):
def__float__(self): ...
reveal_type(Floaty() ifbool() else0)
reveal_type(Supporter() ifbool() else0)

The former is revealed as builtins.object, the latter as typing.SupportsFloat. As short term solution would be to add the base classes to Decimal again, but I believe this should be fixed in mypy.

JukkaL · 2018-07-27T08:21:37Z

@srittau Mypy works like by design, though it's clearly suboptimal in this case. It's not clear how this should be fixed in mypy. It's easy to imagine how this could work in simple cases, but a general solution seems difficult to define -- so that it's both practical and has the desired theoretical properties. Luckily the problems seem to arise pretty infrequently, and they can often be worked around by adding an explicit protocol base class. Can we add the base classes back? We should probably add a comment explaining why they are there so that they won't be removed again in the future.

If you want to propose a mypy fix, I'm happy to discuss it. I've looked into this a bit but gave up as it didn't seem worth the effort at the time.

srittau · 2018-07-27T08:38:45Z

The problem I have with that is that it basically makes protocols useless, because the if .... else ... construct will not work with any protocol. This would mean we'd have to add all protocols to all classes again.

JukkaL · 2018-07-30T10:58:00Z

This is the first time I've seen this cause problems, so it's a bit unfair to say that it makes protocols basically useless. I proposed a potential mypy improvement in python/mypy#5392 (comment), which could partially address the issue.

srittau · 2018-07-30T11:21:14Z

Well, useless in the sense that whenever someone comes up with a case where a protocol is used in an if ... else ... construct, we'd have to add the protocol as base class to the involved classes or add them preemptively. I suspect that it didn't come up much so far is because protocols are not used much yet in typeshed.

But just implementing the suggestion from typing/issues#564 would likely make this problem more pronounced.

* Use Tuple field in DecimalTuple * Remove unnecessary base classes from Decimal * Decimal.__init__ -> __new__ * Decimal.__ne__ is not defined in Python 3 * Add Decimal.as_integer_ratio() * Annotate DecimalException.handle() * Correct types of Decimal method arguments * Add missing arguments and optional markers to Decimal * Add missing arguments to Context * Remove spurious int from Unions with float * Remove Context.__setattr__() * Fix return types of Context methods

srittau added 9 commits July 11, 2018 18:24

Use Tuple field in DecimalTuple
2fb0340

Remove unnecessary base classes from Decimal
8b5cc94

Decimal.__init__ -> __new__
719adb6

Decimal.__ne__ is not defined in Python 3
b20fd31

Add Decimal.as_integer_ratio()
5dfec43

Annotate DecimalException.handle()
6d2a0aa

Correct types of Decimal method arguments
9a26419

Add missing arguments and optional markers to Decimal
6783de7

Add missing arguments to Context
70f991a

JelleZijlstra requested changes Jul 14, 2018
View reviewed changes

srittau added 3 commits July 14, 2018 11:09

Remove spurious int from Unions with float
24ba10f

Remove Context.__setattr__()
37d7614

Fix return types of Context methods
6e26feb

JelleZijlstra closed this Jul 14, 2018

JelleZijlstra reopened this Jul 14, 2018

JelleZijlstra approved these changes Jul 19, 2018
View reviewed changes

JelleZijlstra merged commit 4b8c374 into python:masterJul 19, 2018

srittau deleted the fix-decimal branch July 19, 2018 08:42

srittau mentioned this pull request Jul 26, 2018
Protocols and ... if ... else ...python/mypy#5392
Closed

Uh oh!

Uh oh!

Conversation

srittau commented Jul 11, 2018

Uh oh!

srittau commented Jul 12, 2018

Uh oh!

srittau commented Jul 13, 2018

Uh oh!

JelleZijlstra commented Jul 13, 2018

Uh oh!

JelleZijlstra left a comment

Choose a reason for hiding this comment

Uh oh!

JelleZijlstraJul 14, 2018

Choose a reason for hiding this comment

Uh oh!

JelleZijlstraJul 14, 2018

Choose a reason for hiding this comment

Uh oh!

srittauJul 14, 2018

Choose a reason for hiding this comment

Uh oh!

JelleZijlstraJul 14, 2018

Choose a reason for hiding this comment

Uh oh!

srittauJul 14, 2018

Choose a reason for hiding this comment

Uh oh!

srittau commented Jul 14, 2018

Uh oh!

JelleZijlstra commented Jul 14, 2018

Uh oh!

gvanrossum commented Jul 26, 2018

Uh oh!

srittau commented Jul 26, 2018

Uh oh!

JukkaL commented Jul 27, 2018

Uh oh!

srittau commented Jul 27, 2018

Uh oh!

JukkaL commented Jul 30, 2018

Uh oh!

srittau commented Jul 30, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants