Support python 3.10

[jkevingutierrez]

Trying to install google-python-cloud-debugger with Python 3.10

Results in

ERROR: Could not find a version that satisfies the requirement google-python-cloud-debugger<3.0,>=2.18 (from versions: none)
ERROR: No matching distribution found for google-python-cloud-debugger<3.0,>=2.18

I'm using Docker with python:3.10-slim

[mctavish]

We don't have a release that supports the 3.10 opcodes yet. We'll update this when support is added.

[ermakov-oleg]

@mctavish Is there any information about plans to support python 3.10? When might this happen?

[mctavish]

I ran into some backwards compatibility issues with Python 3.10. I expect that Python 3.10 support will be ready for a release in the next week or so.

[mctavish]

Quick update: pep 626 impacted the debugger in some pretty significant ways. I'm still working on getting things to function properly in Python 3.10

[dvkovner]

@mctavish Hi, James.

I've already tried to fix the issue with Python 3.10. Maybe some of my results could help you somehow:

1. Previously branch operations used an index of a byte as an argument and now they use an index of a bytecode instruction as an argument.
   cPython 3.9:
   #define JUMPTO(x) (next_instr = first_instr + (x) / sizeof(_Py_CODEUNIT))
   #define JUMPBY(x) (next_instr += (x) / sizeof(_Py_CODEUNIT))
   cPython 3.10:
   #define JUMPTO(x) (next_instr = first_instr + (x))
   #define JUMPBY(x) (next_instr += (x))

I've resolved this issue using multiplication and division by 2 (changed lines are highlited with ----------------> and <----------------):

static bool InsertAndUpdateBranchInstructions(
 Insertion insertion, std::vector<UpdatedInstruction>& instructions) {
 std::vector<Insertion> insertions { insertion };
 int iterations = 0;
 while (insertions.size() && iterations < kMaxInsertionIterations) {
 insertion = insertions.back();
 insertions.pop_back();
 // Update the offsets of all insertions after.
 for (auto it = insertions.begin(); it < insertions.end(); it++) {
 if (it->current_offset >= insertion.current_offset) {
 it->current_offset += insertion.size;
 }
 }
 // Update the offsets and arguments of the branches.
 for (auto it = instructions.begin();
 it < instructions.end(); it++) {
 PythonInstruction instruction = it->instruction;
 int32_t arg = static_cast<int32_t>(instruction.argument);
 bool need_to_update = false;
 PythonOpcodeType opcode_type = GetOpcodeType(instruction.opcode);
 if (opcode_type == BRANCH_DELTA_OPCODE) {
 // For relative branches, the argument needs to be updated if the
 // insertion is between the instruction and the target.
 // The Python compiler sometimes prematurely adds EXTENDED_ARG with an
 // argument of 0 even when it is not required. This needs to be taken
 // into account when calculating the target of a branch instruction.
 int inst_size = std::max(instruction.size, it->original_size);
 ---------------->int32_t target = it->current_offset + inst_size + arg * 2;<----------------
 need_to_update = it->current_offset < insertion.current_offset &&
 insertion.current_offset < target;
 } else if (opcode_type == BRANCH_ABSOLUTE_OPCODE) {
 // For absolute branches, the argument needs to be updated if the
 // insertion before the target.
 ---------------->need_to_update = insertion.current_offset < arg * 2;<----------------
 }

 // If we are inserting the original method call instructions, we want to
 // update the current_offset of any instructions at or after. If we are
 // doing an EXTENDED_ARG insertion, we don't want to update the offset of
 // instructions right at the offset, because that is the original
 // instruction that the EXTENDED_ARG is for.
 int offset_diff = it->current_offset - insertion.current_offset;
 if ((iterations == 0 && offset_diff >= 0) || (offset_diff > 0)) {
 it->current_offset += insertion.size;
 }

 if (need_to_update) {
 PythonInstruction new_instruction =
 ---------------->PythonInstructionArg(instruction.opcode, arg + insertion.size / 2);<----------------
 int size_diff = new_instruction.size - instruction.size;
 if (size_diff > 0) {
 insertions.push_back(Insertion { size_diff, it->current_offset });
 }
 it->instruction = new_instruction;
 }
 }
 iterations++;
 }
 return insertions.size() == 0;
}

2. co_lnotab is prohibited to change at the moment. I've tried to realize a replacement of the value of "code" attribute using a totally new code object created with PyCode_New() method, but stuck with a strange behaviour described here: https://stackoverflow.com/questions/71522144/wrong-sequence-of-bytes-after-creating-of-a-new-code-object.

Hope, you will resolve this issue soon!

[dvkovner]

@mctavish Hello. Any news about support of Python 3.10?

[mctavish]

@dvkovner Hi, sorry for going quiet there; there have been some other priorities that have kept me busy. One of them is updating how the agent builds and releases here. Once that's out I'll take another swing at Python 3.10.

Thanks for sharing your findings. They should prove useful.

The other aspect of Python 3.10 that's a problem is that the co_lnotab structure has been removed from the C-API. The current implementation of breakpoints requires modifying that structure when the code is modified to ensure that we still have correct line numbers. The replacement is quite different and is only partially specified, which means that the modifications that we've done prior to Python 3.10 won't be easy to replicate.