feat: Support main_module with bootstrap_impl=system_python

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?

[brandonchinn178]

main_module was added in #2671, but support was only added for bootstrap_impl=script. Despite only supporting bootstrap_impl=script, both the CHANGELOG and the documentation didn't mention that limitation, making it seem like main_module doesn't work out-of-the-box (given that bootstrap_impl=system_python is the default). See #2765 (comment).

This PR adds support for main_module with bootstrap_impl=system_python. This is categorized as feat because it's new functionality, but it can also be conceptualized as a fix, since it's bringing the feature in line with docs.

[rickeylev]

Thanks for the PR and test!

aside: if you grant maintainer permssions to edit, i can directly do some of the small things.

[rickeylev]

This needs to be kept because it ensures the correct entry is added to sys.path. Without it, it can inadvertently rely on the implicit behavior from the flag that adds repos to sys.path and the PYTHONSAFEPATH setting

[brandonchinn178]

I can re-add it, but I don't understand the reasoning. IIUC, imports says "add this directory to PYTHONPATH", but we're specifying main_module with the full absolute path, so imports = ["."] isn't doing anything. Would imports = [] have the behavior you want?

[rickeylev]

Instead of creating a shim like this, modify the bootstrap template. All it should really need is to populate a %main_module% variable and pass -m when invoking python. That should simplify things quite a bit.

[brandonchinn178]

hm I could try it again, but it was difficult because then I'd have to update all the locations where %main% is used, since there isn't a main file anymore. It seemed easier to say "we're using the main file logic everywhere still, it just points to a shim"

[brandonchinn178]

yeah, it seems like the notion of a main file is hardcoded in a lot of places in python_bootstrap_template.txt. It's also not immediately clear how the translation to -m <main_module> would work for the coverage path. The shim seems like the easiest solution, especially given this is a deprecated codepath and it should have a negligible performance impact

[brandonchinn178]

@rickeylev Any updates?