Logically merge cell and locals array. They are already contiguous in memory

[markshannon]

BPO	43693
Nosy	@gvanrossum, @rhettinger, @ncoghlan, @markshannon, @ericsnowcurrently, @serhiy-storchaka, @gvanrossum, @pablogsal
PRs	bpo-43693: Compute deref offsets in compiler #25152 bpo-43693: Group the code in codeobject.c logically. #26216 bpo-43693: Add _PyCode_New() and do some related cleanup. #26258 bpo-43693: Clean up the PyCodeObject fields. #26364 bpo-43693: Add _PyCode_New(). #26375 bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. #26388 bpo-43693: Add the MAKE_CELL opcode and interleave fast locals offsets. #26396 bpo-43693: Revert commits 2c1e2583fdc4db6b43d163239ea42b0e8394171f and b2bf2bc1ece673d387341e06c8d3c2bc6e259747  #26530 bpo-43693: Un-revert commits 2c1e258 and b2bf2bc. #26577 bpo-43693: Eliminate unused "fast locals". #26587 bpo-43693: Silence some compiler warnings. #26588 Revert "bpo-43693: Add the MAKE_CELL opcode and interleave fast locals offsets. (gh-26396)" #26597 bpo-43693: Un-revert commit f3fa63e. #26609 bpo-43693: Do not check co_cell2arg if a non-cell offset. #26626 bpo-43693: Turn localspluskinds into an object #26749 bpo-43693 Get rid of CO_NOFREE -- it's unused #26839

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-04-01.11:33:50.457>
labels = ['interpreter-core', 'performance']
title = 'Logically merge cell and locals array. They are already contiguous in memory'
updated_at = <Date 2021-07-21.18:07:49.033>
user = 'https://github.com/markshannon'

bugs.python.org fields:

activity = <Date 2021-07-21.18:07:49.033>
actor = 'gvanrossum'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2021-04-01.11:33:50.457>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 43693
keywords = ['patch']
message_count = 32.0
messages = ['389974', '390069', '390247', '390253', '394455', '394556', '395014', '395059', '395097', '395098', '395102', '395105', '395137', '395273', '395294', '395297', '395317', '395318', '395319', '395321', '395326', '395329', '395331', '395332', '395338', '395365', '395448', '395899', '396015', '396288', '396433', '397956']
nosy_count = 8.0
nosy_names = ['gvanrossum', 'rhettinger', 'ncoghlan', 'Mark.Shannon', 'eric.snow', 'serhiy.storchaka', 'Guido.van.Rossum', 'pablogsal']
pr_nums = ['25152', '26216', '26258', '26364', '26375', '26388', '26396', '26530', '26577', '26587', '26588', '26597', '26609', '26626', '26749', '26839']
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue43693'
versions = []

Linked PRs

• gh-87859: Track Code Object Local Kinds For Arguments #132980

[markshannon]

In the interpreter and compiler, the "fast" locals array and cells array are treated separately. By merging them in the compiler, the interpreter can be simplified a bit.

[rhettinger]

This doesn't look simpler to me. We will forever lose mental clock cycles disentangling the locals and cells. So, I don't think they should be commingled.

[gvanrossum]

To me it looks simpler though. The locals and cells are already stored in a single array, f_localsplus (which also contains the evaluation stack). There are various complications in ceval.c to translate cell indexes to indexes in this array (ein particular the extra local variable 'freevars', which weighs down the stack frame).

Making the interpreter simpler by moving things to the compiler also makes it easier for the C compiler to optimize the code of the interpreter better.

[serhiy-storchaka]

LGTM.

[markshannon]

New changeset 6cc800d by Eric Snow in branch 'main':
bpo-43693: Clean up the PyCodeObject fields. (GH-26364)
6cc800d

[ericsnowcurrently]

New changeset 9f494d4 by Eric Snow in branch 'main':
bpo-43693: Add _PyCode_New(). (gh-26375)
9f494d4

[ericsnowcurrently]

New changeset 2c1e258 by Eric Snow in branch 'main':
bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)
2c1e258

[ericsnowcurrently]

New changeset b2bf2bc by Mark Shannon in branch 'main':
bpo-43693: Compute deref offsets in compiler (gh-25152)
b2bf2bc

[pablogsal]

Unfortunately, commit 2c1e258 has broken all the refleak buildbots. Example failure:

BISECTION:

c1e2583fdc4db6b43d163239ea42b0e8394171f is the first bad commit
commit 2c1e258
Author: Eric Snow <ericsnowcurrently@gmail.com>
Date: Thu Jun 3 10:28:27 2021 -0600

bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)

https://buildbot.python.org/all/#/builders/384/builds/50/steps/5/logs/stdio

As this is affecting all test, I am proceeding with a revert of commit 2c1e258 directly to not mask other issues.