New backward-compatible NamedTuple

[ilevkivskyi]

@gvanrossum
I implemented your idea:

classEmployee(NamedTuple):
name: strid: int

in a backward-compatible manner. Namely, the same code is triggered on instantiation and on subclassing, so that one can use the new form above as well as Employee = NamedTuple('Employee', [('name', str), ('id', int)]). All old tests pass unmodified.

I didn't backport this to Python 2 since it is not possible to use the new form there anyway.

[ilevkivskyi]

Oops, sorry, didn't check this on some older Python 3. Will fix this in a second.

[ilevkivskyi]

OK, now all green. Sorry for exploitation of your Travis, I don't have so many older Python versions here.

[ilevkivskyi]

@gvanrossum Important note: here I am relying on ordered dict in 3.6+, but I think this is OK, because I need it ordered only for definition order in a class.

[ilevkivskyi]

I am actually so excited now: two cool new features -- ordered dicts and annotations -- allowed to realize this beautiful new NamedTuple in essentially four lines of code, the rest is just backward compatibility stuff. Fantastic!

[gvanrossum]

Very cool to see this! You realize mypy doesn't understand this yet, right? But it's still pretty cool, so I'll push it now. Will you be a dear and prep a diff to copy typing.py back to 3.5 and 3.6?

[ilevkivskyi]

Ok, I will open an issue on b.p.o. and will submit patches there. I think we could just copy the same files to both, since these files work for both 3.5. and 3.6.

[gvanrossum]

OK, then just submit the 3.5 patch, and I'll merge forward.

[ilevkivskyi]

@gvanrossum
Here I opened an issue with the patch for 3.5 http://bugs.python.org/issue28079

[ilevkivskyi]

@gvanrossum

Very cool to see this! You realize mypy doesn't understand this yet, right?

I am glad that you like this :-) Btw, I want to work more on mypy internals, but I would like to try few things before. Namely:

• http://bugs.python.org/issue10544 (binding of yield in comprehensions to the enclosing scope) Yury also seems to be interested in this;
• https://mail.python.org/pipermail/python-dev/2015-October/141993.html (refleak in classes with empty __slots__ because of cycle global instance -> type -> method -> module globals -> global instance);
• fixing Kill __subclasscheck__ #136 here. I almost have a plan here. I will describe it when ready.

Ideally, I would like to do this by 3.6b3 and then to focus on mypy for a month or two.

[gvanrossum]

Btw, I want to work more on mypy internals, but I would like to try few things before. Namely:

• http://bugs.python.org/issue10544 (binding of yield in comprehensions to the enclosing scope) Yury also seems to be interested in this;

But it's just a bug with a confusing diagnostic, right? If it gets less confusing, fine, but hardly a priority.

• https://mail.python.org/pipermail/python-dev/2015-October/141993.html (refleak in classes with empty slots because of cycle global instance -> type -> method -> module globals -> global instance);

I can't actually follow that thread -- isn't there a bug for it?

fixing #136 here. I almost have a plan here. I will describe it when ready.

OK -- but isn't #207 also intending to fix that?

Ideally, I would like to do this by 3.6b3 and then to focus on mypy for a month or two.

That sounds fine -- maybe someone else will beat you to it, but probably not anyone from Dropbox, since we're mostly focused on 2.7 still...

[ilevkivskyi]

@gvanrossum

But it's just a bug with a confusing diagnostic, right? If it gets less confusing, fine, but hardly a priority.

Indeed, I find the current behavior (binding yield inside comprehension auxiliary function scope) confusing. But I understand that this is personal (this is just something that I wanted to work on for a long time, so that I want to do this while I have some time for it).
But more important, in your opinion what are the high priority things to do in core CPython now? Maybe I could also try to do some of those things. I have got very interested in core CPython while implementing PEP 526 (to be honest I was impressed by CPython codebase in terms of quality and ease of work with it).

I can't actually follow that thread -- isn't there a bug for it?

Unfortunately, there is no separate issue for this yet, it was discovered in http://bugs.python.org/issue24379 , but again I don't think it is something high priority in general, it is rather interesting for me.

OK -- but isn't #207 also intending to fix that?

Yes, but I would like to propose an alternative way, by making most thing like Union etc. not actual classes, just objects that record they parameters. This is how Mark Shannon proposed to do this some time ago, I tend to agree with him, this way will solve __subclasscheck__ problem, but also some other problems and will give a significant speed-up (not for Generic[T] although, these must be classes). But, I propose to postpone this discussion until I will make a detailed plan (or even a PR) for this.