Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork 35.2k
bpo-36540: PEP 570 -- Implementation#12701
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
Changes from all commits
d2d00cb09e64c6734b22003c85f873ac7b76d2eb5a7d3240147efe56d5c86eb5200f68e13be2521669a1cd8c944588a3d1f5f1952791c44c5dbaed3afe86b90514954File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Uh oh!
There was an error while loading. Please reload this page.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -272,6 +272,7 @@ def iscode(object): | ||
| | 16=nested | 32=generator | 64=nofree | 128=coroutine | ||
| | 256=iterable_coroutine | 512=async_generator | ||
| co_freevars tuple of names of free variables | ||
| co_posonlyargcount number of positional only arguments | ||
| co_kwonlyargcount number of keyword only arguments (not including ** arg) | ||
| co_lnotab encoded mapping of line numbers to bytecode indices | ||
| co_name name with which this code object was defined | ||
| @@ -1031,26 +1032,20 @@ def getargs(co): | ||
| 'args' is the list of argument names. Keyword-only arguments are | ||
| appended. 'varargs' and 'varkw' are the names of the * and ** | ||
| arguments or None.""" | ||
| args, varargs, kwonlyargs, varkw = _getfullargs(co) | ||
| return Arguments(args + kwonlyargs, varargs, varkw) | ||
| def _getfullargs(co): | ||
| """Get information about the arguments accepted by a code object. | ||
| Four things are returned: (args, varargs, kwonlyargs, varkw), where | ||
| 'args' and 'kwonlyargs' are lists of argument names, and 'varargs' | ||
| and 'varkw' are the names of the * and ** arguments or None.""" | ||
| if not iscode(co): | ||
| raise TypeError('{!r} is not a code object'.format(co)) | ||
| nargs = co.co_argcount | ||
| names = co.co_varnames | ||
| nargs = co.co_argcount | ||
| nposonlyargs = co.co_posonlyargcount | ||
| nkwargs = co.co_kwonlyargcount | ||
| args = list(names[:nargs]) | ||
| kwonlyargs = list(names[nargs:nargs+nkwargs]) | ||
| nposargs = nargs + nposonlyargs | ||
| posonlyargs = list(names[:nposonlyargs]) | ||
| args = list(names[nposonlyargs:nposonlyargs+nargs]) | ||
| kwonlyargs = list(names[nposargs:nposargs+nkwargs]) | ||
| step = 0 | ||
| nargs += nposonlyargs | ||
| nargs += nkwargs | ||
| varargs = None | ||
| if co.co_flags & CO_VARARGS: | ||
| @@ -1059,8 +1054,7 @@ def _getfullargs(co): | ||
| varkw = None | ||
| if co.co_flags & CO_VARKEYWORDS: | ||
| varkw = co.co_varnames[nargs] | ||
| return args, varargs, kwonlyargs, varkw | ||
| return Arguments(posonlyargs + args + kwonlyargs, varargs, varkw) | ||
| ArgSpec = namedtuple('ArgSpec', 'args varargs keywords defaults') | ||
| @@ -1087,15 +1081,16 @@ def getargspec(func): | ||
| warnings.warn("inspect.getargspec() is deprecated since Python 3.0, " | ||
| "use inspect.signature() or inspect.getfullargspec()", | ||
| DeprecationWarning, stacklevel=2) | ||
| args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, ann = \ | ||
| getfullargspec(func) | ||
| if kwonlyargs or ann: | ||
| raise ValueError("Function has keyword-only parameters or annotations" | ||
| ", use getfullargspec() API which can support them") | ||
| args, varargs, varkw, defaults, posonlyargs, kwonlyargs, \ | ||
| kwonlydefaults, ann = getfullargspec(func) | ||
| if posonlyargs or kwonlyargs or ann: | ||
| raise ValueError("Function has positional-only, keyword-only parameters" | ||
| " or annotations, use getfullargspec() API which can" | ||
| " support them") | ||
| return ArgSpec(args, varargs, varkw, defaults) | ||
| FullArgSpec = namedtuple('FullArgSpec', | ||
| 'args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, annotations') | ||
| 'args, varargs, varkw, defaults, posonlyargs, kwonlyargs, kwonlydefaults, annotations') | ||
| def getfullargspec(func): | ||
| """Get the names and default values of a callable object's parameters. | ||
| @@ -1145,6 +1140,7 @@ def getfullargspec(func): | ||
| args = [] | ||
| varargs = None | ||
| varkw = None | ||
| posonlyargs = [] | ||
| kwonlyargs = [] | ||
| defaults = () | ||
| annotations = {} | ||
| @@ -1159,7 +1155,9 @@ def getfullargspec(func): | ||
| name = param.name | ||
| if kind is _POSITIONAL_ONLY: | ||
| args.append(name) | ||
| posonlyargs.append(name) | ||
| if param.default is not param.empty: | ||
| defaults += (param.default,) | ||
| elif kind is _POSITIONAL_OR_KEYWORD: | ||
| args.append(name) | ||
| if param.default is not param.empty: | ||
| @@ -1185,7 +1183,7 @@ def getfullargspec(func): | ||
| defaults = None | ||
| return FullArgSpec(args, varargs, varkw, defaults, | ||
Member There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is breaking change. MemberAuthor There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suspect that there may be a longer discussion for these cases so I opened https://bugs.python.org/issue36751 to track it and decide there. | ||
| kwonlyargs, kwdefaults, annotations) | ||
| posonlyargs, kwonlyargs, kwdefaults, annotations) | ||
| ArgInfo = namedtuple('ArgInfo', 'args varargs keywords locals') | ||
| @@ -1216,7 +1214,8 @@ def _formatannotation(annotation): | ||
| return _formatannotation | ||
| def formatargspec(args, varargs=None, varkw=None, defaults=None, | ||
| kwonlyargs=(), kwonlydefaults={}, annotations={}, | ||
| posonlyargs=(), kwonlyargs=(), kwonlydefaults={}, | ||
Member There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since all these parameters are positional-or-keyword, inserting a new parameter in the middle is breaking change. You can add it only to the end. Or add a new function. MemberAuthor There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suspect that there may be a longer discussion for these cases so I opened https://bugs.python.org/issue36751 to track it and decide there. | ||
| annotations={}, | ||
| formatarg=str, | ||
| formatvarargs=lambda name: '*' + name, | ||
| formatvarkw=lambda name: '**' + name, | ||
| @@ -1249,12 +1248,17 @@ def formatargandannotation(arg): | ||
| return result | ||
| specs = [] | ||
| if defaults: | ||
| firstdefault = len(args) - len(defaults) | ||
| for i, arg in enumerate(args): | ||
| firstdefault = len(posonlyargs) + len(args) - len(defaults) | ||
| posonly_left = len(posonlyargs) | ||
| for i, arg in enumerate([*posonlyargs, *args]): | ||
| spec = formatargandannotation(arg) | ||
| if defaults and i >= firstdefault: | ||
| spec = spec + formatvalue(defaults[i - firstdefault]) | ||
| specs.append(spec) | ||
| posonly_left -= 1 | ||
| if posonlyargs and posonly_left == 0: | ||
| specs.append('/') | ||
| if varargs is not None: | ||
| specs.append(formatvarargs(formatargandannotation(varargs))) | ||
| else: | ||
| @@ -1342,7 +1346,8 @@ def getcallargs(*func_and_positional, **named): | ||
| func = func_and_positional[0] | ||
| positional = func_and_positional[1:] | ||
| spec = getfullargspec(func) | ||
| args, varargs, varkw, defaults, kwonlyargs, kwonlydefaults, ann = spec | ||
| (args, varargs, varkw, defaults, posonlyargs, | ||
| kwonlyargs, kwonlydefaults, ann) = spec | ||
| f_name = func.__name__ | ||
| arg2value = {} | ||
| @@ -1351,12 +1356,16 @@ def getcallargs(*func_and_positional, **named): | ||
| # implicit 'self' (or 'cls' for classmethods) argument | ||
| positional = (func.__self__,) + positional | ||
| num_pos = len(positional) | ||
| num_posonlyargs = len(posonlyargs) | ||
| num_args = len(args) | ||
| num_defaults = len(defaults) if defaults else 0 | ||
| n = min(num_pos, num_posonlyargs) | ||
| for i in range(num_posonlyargs): | ||
| arg2value[posonlyargs[i]] = positional[i] | ||
| n = min(num_pos, num_args) | ||
| for i in range(n): | ||
| arg2value[args[i]] = positional[i] | ||
| arg2value[args[i]] = positional[num_posonlyargs+i] | ||
| if varargs: | ||
| arg2value[varargs] = tuple(positional[n:]) | ||
| possible_kwargs = set(args + kwonlyargs) | ||
| @@ -2137,9 +2146,12 @@ def _signature_from_function(cls, func): | ||
| func_code = func.__code__ | ||
| pos_count = func_code.co_argcount | ||
| arg_names = func_code.co_varnames | ||
| positional = tuple(arg_names[:pos_count]) | ||
| posonly_count = func_code.co_posonlyargcount | ||
| positional_count = posonly_count + pos_count | ||
| positional_only = tuple(arg_names[:posonly_count]) | ||
| positional = tuple(arg_names[posonly_count:positional_count]) | ||
| keyword_only_count = func_code.co_kwonlyargcount | ||
| keyword_only = arg_names[pos_count:(pos_count + keyword_only_count)] | ||
| keyword_only = arg_names[positional_count:(positional_count + keyword_only_count)] | ||
| annotations = func.__annotations__ | ||
| defaults = func.__defaults__ | ||
| kwdefaults = func.__kwdefaults__ | ||
| @@ -2151,23 +2163,33 @@ def _signature_from_function(cls, func): | ||
| parameters = [] | ||
| non_default_count = positional_count - pos_default_count | ||
| all_positional = positional_only + positional | ||
| posonly_left = posonly_count | ||
| # Non-keyword-only parameters w/o defaults. | ||
| non_default_count = pos_count - pos_default_count | ||
| for name in positional[:non_default_count]: | ||
| for name in all_positional[:non_default_count]: | ||
| kind = _POSITIONAL_ONLY if posonly_left else _POSITIONAL_OR_KEYWORD | ||
| annotation = annotations.get(name, _empty) | ||
| parameters.append(Parameter(name, annotation=annotation, | ||
| kind=_POSITIONAL_OR_KEYWORD)) | ||
| kind=kind)) | ||
| if posonly_left: | ||
| posonly_left -= 1 | ||
| # ... w/ defaults. | ||
| for offset, name in enumerate(positional[non_default_count:]): | ||
| for offset, name in enumerate(all_positional[non_default_count:]): | ||
| kind = _POSITIONAL_ONLY if posonly_left else _POSITIONAL_OR_KEYWORD | ||
| annotation = annotations.get(name, _empty) | ||
| parameters.append(Parameter(name, annotation=annotation, | ||
| kind=_POSITIONAL_OR_KEYWORD, | ||
| kind=kind, | ||
| default=defaults[offset])) | ||
| if posonly_left: | ||
| posonly_left -= 1 | ||
| # *args | ||
| if func_code.co_flags & CO_VARARGS: | ||
| name = arg_names[pos_count + keyword_only_count] | ||
| name = arg_names[positional_count + keyword_only_count] | ||
| annotation = annotations.get(name, _empty) | ||
| parameters.append(Parameter(name, annotation=annotation, | ||
| kind=_VAR_POSITIONAL)) | ||
| @@ -2184,7 +2206,7 @@ def _signature_from_function(cls, func): | ||
| default=default)) | ||
| # **kwargs | ||
| if func_code.co_flags & CO_VARKEYWORDS: | ||
| index = pos_count + keyword_only_count | ||
| index = positional_count + keyword_only_count | ||
| if func_code.co_flags & CO_VARARGS: | ||
| index += 1 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -619,8 +619,9 @@ def replace_paths_in_code(self, co): | ||
| if isinstance(consts[i], type(co)): | ||
| consts[i] = self.replace_paths_in_code(consts[i]) | ||
| return types.CodeType(co.co_argcount, co.co_kwonlyargcount, | ||
| co.co_nlocals, co.co_stacksize, co.co_flags, | ||
| return types.CodeType(co.co_argcount, co.co_posonlyargcount, | ||
pablogsal marked this conversation as resolved.
Outdated
Uh oh!There was an error while loading. Please reload this page. | ||
| co.co_kwonlyargcount, co.co_nlocals, | ||
| co.co_stacksize, co.co_flags, | ||
| co.co_code, tuple(consts), co.co_names, | ||
| co.co_varnames, new_filename, co.co_name, | ||
| co.co_firstlineno, co.co_lnotab, co.co_freevars, | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding a required argument to the code type constructor will breaking every library that creates
types.CodeTypeinstances. Are you sure this is a good idea?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like it has already started: https://www.google.com/search?q=TypeError%3A+code%28%29+%22takes+at+least+14+arguments+%2813+given%29%22
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You missed the end of the story:
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also,
types.CodeType's constructor is considered private. We now even have this warning in the docs (https://docs.python.org/3.8/library/types.html#standard-interpreter-types):Additionally, you have now available
types.CodeType.replacein case you can to create a copy with some fields changed.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the explanation!
I hope in future we'll have a more stable API for dynamic function generation.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This breaks
pickle.loadsfor functions constructed asand pickled using
cloudpickleunder Python 3.7; these can't be loaded back up under Python 3.8.Should this be addressed downstream in cloudpickle ?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, they need to update whatever code they are using to create code objects as code objects in Python3.8 require an extra field. Please, check these sections in the docs:
https://docs.python.org/3/library/types.html#standard-interpreter-types
Also, check this new C-API function:
https://docs.python.org/3/c-api/code.html#c.PyCode_NewWithPosOnlyArgs
and this new convenience function:
https://docs.python.org/3/library/types.html#types.CodeType.replace
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, I'll file an issue there. Thanks.