Uh oh!
There was an error while loading. Please reload this page.
Don't include pyc files as part of the hermetic toolchain - #713
Conversation
pyc files don't appear to be generated deterministically, so including them as part of the :files filegroup means that actions that use the hermetic toolchain do not have deterministic hashes, and so bazel needlessly re-executes them. This can be seen when using bazel together with a remote cache: builds on separate machines will generate pyc files with different hashes, leading to cache misses.
tomgr
commented
May 31, 2022
No problem - thanks for merging! |
f0rmiga
left a comment
There was a problem hiding this comment.
I first would like to understand why this happens before simply excluding .pyc files. To clarify, we download the python toolchain deterministically, then make the directory tree read-only. So, excluding the .pyc sounds like burying the real problem.
f0rmiga
commented
May 31, 2022
@tomgr You said you see this on all platforms, right? The hash you posted is from the Windows toolchain, which we don't make the tree read-only. |
f0rmiga
commented
May 31, 2022
f0rmiga
commented
May 31, 2022
Sorry, 43bdb71 should be the one... |
tomgr
commented
May 31, 2022
I've just tried using 43bdb71 and it still has the same problem. And just to confirm: we saw this on all three platforms, and this patch appears to have completely solved the issue. We've been using it for all of our CI runs for the last week, and our builds now complete entirely from the cache if there are no changes. Previously, whenever we had a fresh VM bazel would rebuild all of the targets that used python. As an example, with your latest patch, I again saw the hash of and on a second run: The same was also true on windows with the first run containing: and the second run: It seems that not all python modules suffer from the problem. Looking at the logs I see the issue with: gettext, markupbase, typing, pathlib, socket, site, zipfile, hashlib and message, but not any others, despite the fact we do use more modules than that. There's no obvious pattern from this. I presume that the read-only permissions must be being overridden. We do run our Linux builds inside a docker container with the repository cache mounted as a volume. However our Windows and Mac builds run directly on a VM and we still see the problem on there: each different VM creates different pyc files. We do run bazel using a specific |
f0rmiga
commented
Jun 1, 2022
@tomgr When you say "first run" and then "second run", is there any |
It throws me off that somehow read-only is not taking any effects. |
tomgr
commented
Jun 1, 2022
There's no Is there any extra data that I can gather to help debug why the read-only setting is not working? |
mattoberle
commented
Jun 6, 2022
@tomgr any chance your builds are running as |
alexgartner-bc
commented
Jul 1, 2022
Interesting. I'm seeing similar issues but only on our stateless build servers (which are running builds as root) vs our stateful build servers which run builds as uid 1000. This only started happening after we configured container-diff diffSounds like an interim solution might be |
A better idea is not to run as root! |
I generally agree but we are currently using google cloud build which does not allow you to run as non-root (afaik). Just fyi for anyone else browsing this thread later. Update: actually there is a way |
phlax
commented
Aug 15, 2022
i use a temporary container locally for working on a project using i get this is not "best-practice" but im well aware of the risks and in this case there arent really any it seems a bit arbitrary to enforce that you cant use |
Cartman75
commented
Sep 9, 2022
BTw, its seems impossible to run this in gitlab because it forces root. Did someone figure out how to get around that? I even tried: and claimed it was it running as root and it was a no-go. |
keith
commented
Sep 23, 2022
FYI there is a setting to opt out of this root warning, https://github.com/bazelbuild/rules_python//commit/e67e7dd719d34d5a13c15b24d0234c1ac753b52d |
debkanchan
commented
Jan 17, 2024
This is causing GCP cloud build to fail |
layus
commented
Jun 25, 2024
Did anyone ever consider and/or test |
This is a requirement for one dependency. Otherwise, we get the error: "The current user is root, please run as non-root when using the hermetic Python interpreter. See bazel-contrib/rules_python#713."
This is a requirement for one dependency. Otherwise, we get the error: "The current user is root, please run as non-root when using the hermetic Python interpreter. See bazel-contrib/rules_python#713."
This is a requirement for one dependency. Otherwise, we get the error: "The current user is root, please run as non-root when using the hermetic Python interpreter. See bazel-contrib/rules_python#713."
loeffel-io
commented
Nov 19, 2024
Hey, for clarification I do need to add this python=use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
configure_coverage_tool=False,
ignore_root_user_error=True,
python_version="3.11",
)while one of my deps is using it works btw but this does not feel good Thanks |
Our CI fails to build Bazel, producing this error: "The current user is root, please run as non-root when using the hermetic Python interpreter. See bazel-contrib/rules_python#713." The workaround was suggested at bazel-contrib/rules_python#1169 (comment)
Our CI fails to build Bazel, producing this error: "The current user is root, please run as non-root when using the hermetic Python interpreter. See bazel-contrib/rules_python#713." The workaround was suggested at bazel-contrib/rules_python#1169 (comment)Closes#24549. PiperOrigin-RevId: 702362811 Change-Id: Ib6d4da1e09fd2b7dfd68af02bbb0eb997e8f427c
Our CI fails to build Bazel, producing this error: "The current user is root, please run as non-root when using the hermetic Python interpreter. See bazel-contrib/rules_python#713." The workaround was suggested at bazel-contrib/rules_python#1169 (comment)Closes#24549. PiperOrigin-RevId: 702362811 Change-Id: Ib6d4da1e09fd2b7dfd68af02bbb0eb997e8f427c Co-authored-by: Florian Weikert <fwe@google.com>
Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes#1169.
Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes#1169.
Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes#1169.
Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes#1169.
Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes#1169.
Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes#1169. --------- Co-authored-by: Richard Levasseur <rlevasseur@google.com>
…-contrib#2636) Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at bazel-contrib#713, bazel-contrib#749, bazel-contrib#907, bazel-contrib#1008, bazel-contrib#1169, etc. Fixesbazel-contrib#1169. --------- Co-authored-by: Richard Levasseur <rlevasseur@google.com>
This leads to errors, see bazel-contrib/rules_python#713 (comment)
PR Checklist
Please check if your PR fulfills the following requirements:
I'm not sure how to test this.
PR Type
What kind of change does this PR introduce?
What is the current behavior?
Using rules_python with the hermetic toolchain via
python_register_toolchainsresults in cache misses when using bazel with a remote cache. This appears to be because pyc files that are generated on the build host are included as part of one of the globs, but since pyc files don't appear to be generated deterministically. For example, from one run we saw one action that used the hermetic toolchain had the following file as an input:but from another run we saw:
The hashes above are taken from the bazel execution log as per https://bazel.build/docs/remote-execution-caching-debug.
Note that although the above are with the windows toolchain, we saw the same behaviour on all MacOS, Linux, and Windows.
What is the new behavior?
pyc files are excluded from the glob. We're no longer seeing cache misses when using a cache.
Does this PR introduce a breaking change?