Emit slices as constants in the bytecode compiler

[mdboom]

Feature or enhancement

Proposal:

Constant slices are currently emitted as multiple bytecodes by the Python bytecode compiler.

For example, x[:-2, 1:-1] generates:

 LOAD_CONST 4 (0)
 LOAD_CONST 5 (-2)
 BUILD_SLICE 2
 LOAD_CONST 1 (1)
 LOAD_CONST 2 (-1)
 BUILD_SLICE 2
 BUILD_TUPLE 2

Then tuples of slices are constant like this, this could instead be compiled down to a single LOAD_CONST instruction:

 LOAD_CONST 1 ((slice(0, -2, None), slice(1, -1, None))

According to @fberlakovich and Stefan Brunthaler's paper on Cross module quickening, this can have a significant impact on Numpy benchmarks.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-125063: Emit slices as constants in the bytecode compiler #125064
• gh-125063: Document that slices can be marshalled #126804
• gh-125063: marshal: Add version 5, improve documentation #126829
• gh-125063: Move slice constant-folding to AST #126830

[nineteendo]

Shouldn't this be closed now?

[mdboom]

Shouldn't this be closed now?

Yep -- sorry, I forgot that step.

[encukou]

Hi @mdboom,
This is missing a documentation update, since marshal docs have a list of the types that pyc can handle.

But, I wanted to ask: would it make sense to do the folding in AST, rather than in codegen? That seems like a more appropriate place, even though it probably won't help common code (e.g. folding "abc"[:1] to "a" isn't too relevant).

I quickly tried moving it, and tests pass, but I don't know if I'm missing something: main...encukou:cpython:const_slice

[mdboom]

his is missing a documentation update, since marshal docs have a list of the types that pyc can handle.

Thanks for pointing that out. I'll handle that part.

But, I wanted to ask: would it make sense to do the folding in AST, rather than in codegen?

I don't know if I have enough historical to say which is better, it does seem like doing it earlier makes sense. I'd want a second opinion about whether it's the right thing to do, though (Cc: @iritkatriel?), and if so, you should submit your branch as a PR.

[iritkatriel]

But, I wanted to ask: would it make sense to do the folding in AST, rather than in codegen?

Yes, I think this makes sense.