Skip to content

Use real rules_python - #17545

Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom
comius:use-py_proto_library
Closed

Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom
comius:use-py_proto_library

Conversation

@comius

@comiuscomius commented Feb 21, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Initial motivation was to use py_proto_library from rules_python, but then a yak came along.

Fixes: #9029

@comius
comiusforce-pushed the use-py_proto_library branch from 47fd43e to 5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed the use-py_proto_library branch from 11d9548 to 9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius marked this pull request as ready for review February 24, 2023 12:43
@comiuscomius self-assigned this Feb 24, 2023
@meteorcloudy

meteorcloudy commented Feb 24, 2023

Copy link
Copy Markdown
Member

This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?

@comius

comius commented Feb 24, 2023

Copy link
Copy Markdown
ContributorAuthor

Hey, @meteorcloudy,
I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?

Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log

Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test

@meteorcloudy

meteorcloudy commented Feb 24, 2023•
edited
Loading

Copy link
Copy Markdown
Member

It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled

@meteorcloudy

meteorcloudy commented Feb 24, 2023

Copy link
Copy Markdown
Member

I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:

[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy

meteorcloudy commented Feb 24, 2023

Copy link
Copy Markdown
Member

I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:

pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy

meteorcloudy commented Feb 24, 2023

Copy link
Copy Markdown
Member

Do we expect users to have protobuf pip package preinstalled when using py_proto_library?

@sgowrojisgowroji added the team-Rules-Python Native rules for Python label Feb 24, 2023
@ted-xie

ted-xie commented Feb 24, 2023

Copy link
Copy Markdown
Contributor

android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.

@rickeylev

rickeylev commented Feb 24, 2023

Copy link
Copy Markdown
Contributor

Do we expect users to have protobuf pip package preinstalled when using py_proto_library?

No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.

To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)

@comius

comius commented Feb 24, 2023

Copy link
Copy Markdown
ContributorAuthor

Thank @meteorcloudy for the analysis, it saved my day!

I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)

@comius

comius commented Feb 24, 2023

Copy link
Copy Markdown
ContributorAuthor

Ups, I missed comment from @rickeylev. My fix is not actually needed.

@comiuscomius mentioned this pull request Feb 24, 2023
Closed
3 tasks
@comiuscomius added the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Feb 24, 2023
@sgowroji

sgowroji commented Feb 27, 2023

Copy link
Copy Markdown
Member

Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!

@comius

comius commented Feb 27, 2023

Copy link
Copy Markdown
ContributorAuthor

Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!

Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.

@sgowrojisgowroji removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Mar 6, 2023
@fweikertfweikert mentioned this pull request Mar 23, 2023
Closed
comius added a commit to comius/rules_python that referenced this pull request Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius mentioned this pull request Apr 17, 2023
Merged
rickeylev referenced this pull request Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit to werkt/buildfarm that referenced this pull request Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt mentioned this pull request Apr 22, 2023
Closed
werkt added a commit to werkt/buildfarm that referenced this pull request Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit to werkt/buildfarm that referenced this pull request Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit to bazel-contrib/rules_python that referenced this pull request Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev mentioned this pull request Apr 24, 2023
Closed
fweikert pushed a commit to fweikert/bazel that referenced this pull request May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Use real rules_python by comius · Pull Request #17545 · bazelbuild/bazel · GitHub
Skip to content
Use real rules_python - #17545
Closed
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Closed
Use real rules_python#17545
comius wants to merge 12 commits into
bazelbuild:masterfrom 
comius:use-py_proto_library
Conversation
@comius
@comiuscomius
commented
Feb 21, 2023•

edited

Loading

Copy link


Copy Markdown

Contributor
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: #9029
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
47fd43e to
5fcef16CompareFebruary 22, 2023 14:07
@comius
comiusforce-pushed
the
use-py_proto_library
branch
from
11d9548 to
9fd53ebCompareFebruary 24, 2023 10:57
@comius
comius
marked this pull request as ready for review
February 24, 2023 12:43
@comiuscomius
self-assigned this
Feb 24, 2023
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
This is great, I guess we can delete https://github.com/bazelbuild/bazel/tree/master/third_party/rules_python after merging this one?
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Hey, @meteorcloudy,

I can't reproduce the failure in //tools/aquery_differ:aquery_differ_test locally. Could this possibly be a caching issue?
Fails:
bazel test --config=ubuntu1804_java11 -- //tools/aquery_differ:aquery_differ_test, log
Succeeds:
bazel test -- //tools/aquery_differ:aquery_differ_test
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023•

edited

Loading

Copy link


Copy Markdown

Member
It doesn't look like a caching issue, did you try to reproduce in the docker container? This is also failing on other platforms without RBE enabled
@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I can reproduce the issue in the docker container (docker run -it gcr.io/bazel-public/centos7-java11-devtoolset10) but not on my linux workstation:
[root@06cbde8452c1 bazel]# bazel test //tools/aquery_differ:aquery_differ_test
2023/02/24 13:07:54 Downloading https://releases.bazel.build/6.0.0/release/bazel-6.0.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //tools/aquery_differ:aquery_differ_test (63 packages loaded, 1111 targets configured).
INFO: Found 1 test target...
FAIL: //tools/aquery_differ:aquery_differ_test (see /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log)
Target //tools/aquery_differ:aquery_differ_test up-to-date:
bazel-bin/tools/aquery_differ/aquery_differ_test
INFO: Elapsed time: 52.583s, Critical Path: 14.12s
INFO: 208 processes: 7 internal, 201 processwrapper-sandbox.
INFO: Build completed, 1 test FAILED, 208 total actions
//tools/aquery_differ:aquery_differ_test FAILED in 0.2s
/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
Executed 1 out of 1 test: 1 fails locally.
[root@06cbde8452c1 bazel]# /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
[root@06cbde8452c1 bazel]# cat /root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/execroot/io_bazel/bazel-out/k8-fastbuild/testlogs/tools/aquery_differ/aquery_differ_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //tools/aquery_differ:aquery_differ_test
-----------------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/tools/aquery_differ/aquery_differ_test.py", line 23, in <module>
from src.main.protobuf import analysis_v2_pb2
File "/root/.cache/bazel/_bazel_root/c564d33b2d1d9ff3bd9f69877f355ccc/sandbox/processwrapper-sandbox/200/execroot/io_bazel/bazel-out/k8-fastbuild/bin/tools/aquery_differ/aquery_differ_test.runfiles/io_bazel/src/main/protobuf/analysis_v2_pb2.py", line 5, in <module>
from google.protobuf.internal import builder as _builder
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
I guess this is because I do have the protobuf pip package installed on my workstation, but not in the docker container:
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ python3
Python 3.10.9 (main, Dec 7 2022, 13:47:07) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> exit()
pcloudy@pcloudy:~/workspace/bazel (use-py_proto_library)
$ docker exec -it 06cbde8452c1 /bin/bash
[root@06cbde8452c1 /]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'google'

@meteorcloudy
meteorcloudy
commented
Feb 24, 2023

Copy link


Copy Markdown

Member
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
@sgowrojisgowroji
added
the 
team-Rules-Python
Native rules for Python
label
Feb 24, 2023
@ted-xie
ted-xie
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
android-related changes LGTM (just //tools/python -> @rules_python replacement for tools/android/BUILD and tools/android/BUILD.tools). As long as presubmit passes then it should be fine.
@rickeylev
rickeylev
commented
Feb 24, 2023

Copy link


Copy Markdown

Contributor
Do we expect users to have protobuf pip package preinstalled when using py_proto_library?
No, that's a bug that was fixed but hasn't been released yet (see bazel-contrib/rules_python#1046). I'll do a 0.19 release Monday.
To workaround it in the meantime...I'm not sure there's an easy way in the mock environment. The files are there in runfiles, the path to them just has to get onto sys.path. So either manually modify sys.path at runtime, or, maybe manually depend on the proto repo's py_library target (the bug causing the missing sys.path entry is in py_proto_library, so anything coming through that won't propagate the necessary sys.path entry)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Thank @meteorcloudy for the analysis, it saved my day!
I found out there is a bug, in py_proto_library. I worked around it and submitted a patch to rules_python. (bazel-contrib/rules_python#1091)
@comius
comius
commented
Feb 24, 2023

Copy link


Copy Markdown

ContributorAuthor
Ups, I missed comment from @rickeylev. My fix is not actually needed.
@comiuscomius
mentioned this pull request
Feb 24, 2023
 Closed

3 tasks
@comiuscomius
added
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Feb 24, 2023
@sgowroji
sgowroji
commented
Feb 27, 2023

Copy link


Copy Markdown

Member
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
@comius
comius
commented
Feb 27, 2023

Copy link


Copy Markdown

ContributorAuthor
Hi @comius, Could you please guide, Whether to import third-party changes first or the other changes in the CL. Thanks!
Hey, I didn't realize before, but the import is more complex than I thought. I'll prepare separate parts for it.
@sgowrojisgowroji
removed
the 
awaiting-PR-merge
PR has been approved by a reviewer and is ready to be merge internally
label
Mar 6, 2023
@fweikertfweikert
mentioned this pull request
Mar 23, 2023

Closed

comius added a commit
to comius/rules_python
that referenced
this pull request
Apr 17, 2023
@comius
Remove reference to @bazel_tools//tools/python/private:defs.bzl
e8863d6
The file was removed in Bazel@HEAD in bazelbuild/bazel#17545
Addresses: bazelbuild/bazel#17874
@comiuscomius
mentioned this pull request
Apr 17, 2023
 Merged

rickeylev referenced
this pull request
Apr 18, 2023
@rickeylev@copybara-github
python: Flip --incompatible_python_disable_py2 to true
95a57ef
This changes the default to rejecting Python 2 values. For more
information, see #17293Fixes#17293
RELNOTES[INC]: --incompatible_python_disable_py2 is flipped to true. See #17293 for details.
PiperOrigin-RevId: 507897246
Change-Id: I67cbadbb0f543d8153ad0355af22e48ead9083ef
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
7db7d70
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
@werktwerkt
mentioned this pull request
Apr 22, 2023
 Closed

werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
a03815c
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
werkt added a commit
to werkt/buildfarm
that referenced
this pull request
Apr 22, 2023
@werkt
Incorporate rules_{python,proto} for bazel_tools
cdd8c12
Since bazelbuild/bazel#17545, bazel_tools apparantly requires
rules_python and rules_proto to successfully parse under bazel (load
lines only). We only use affected bazel_tools (src/main/protobuf) in
persistentworkers. Import these repositories to reinstate downstream CI
passing.
Ignore rules_oss_audit setup and invoke install_deps omitted from it in
a generated.bzl (move maven here as well). Relocated all WORKSPACE-scope
invocations to deps/defs/generated.
rickeylev pushed a commit
to bazel-contrib/rules_python
that referenced
this pull request
Apr 24, 2023
@comius
fix: remove reference to @bazel_tools//tools/python/private:defs.bzl (#…
73aec8f
…1173)
The file was removed in Bazel@HEAD in
bazelbuild/bazel#17545
This fixes failures when using rules_python with Bazel@HEAD.
Addresses: bazelbuild/bazel#17874
@rickeylevrickeylev
mentioned this pull request
Apr 24, 2023

Closed

fweikert pushed a commit
to fweikert/bazel
that referenced
this pull request
May 25, 2023
@comius@fweikert
Use real rules_python
d4deb10
Initial motivation was to use py_proto_library from rules_python, but then a yak came along.
Fixes: bazelbuild#9029Closesbazelbuild#17545.
PiperOrigin-RevId: 513834100
Change-Id: I11a99381e1169a9fb7a7a3eaa733ddd348ebac2b
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers

@meteorcloudymeteorcloudymeteorcloudy approved these changes
@ted-xieted-xieted-xie approved these changes
@rickeylevrickeylevrickeylev approved these changes
@ahumeskyahumeskyAwaiting requested review from ahumeskyahumesky is a code owner

Assignees

@comiuscomius

Labels

team-Rules-PythonNative rules for Python

Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

Make Bazel itself depend on @rules_python

5 participants

@comius@meteorcloudy@ted-xie@rickeylev@sgowroji