feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aignas merged 5 commits into from
Dec 31, 2024
Merged

feat: add pyi attributes/fields, original source fields #2538

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d4fcb41
feat: add PyInfo fields for storing original sources of a target
rickeylev Nov 28, 2024
18a64db
feat: add pyi_srcs, pyi_deps attributes
rickeylev Dec 28, 2024
b5f9151
update changelog
rickeylev Dec 29, 2024
6fef69c
run precommit
rickeylev Dec 29, 2024
bd9023e
fix test
rickeylev Dec 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
* 3.11.11
* 3.12.8
* 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
only dependencies added. See {obj}`py_library.pyi_srcs` and
`py_library.pyi_deps` (and the same named attributes for `py_binary` and
`py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
{obj}`direct_original_sources`, {obj}`direct_pyi_files`,
{obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down
29 changes: 29 additions & 0 deletions python/private/attributes.bzl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
),
"pyi_deps": attr.label_list(
doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).

@aignasaignasDec 31, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.

  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?


:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
providers = [
[PyInfo],
[CcInfo],
] + _MaybeBuiltinPyInfo,
),
"pyi_srcs": attr.label_list(
doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
allow_files = True,
),
# Required attribute, but details vary by rule.
# Use create_srcs_attr to create one.
"srcs": None,
Expand Down
11 changes: 10 additions & 1 deletion python/private/common.bzl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
ctx,
*,
original_sources,
required_py_files,
required_pyc_files,
implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

Args:
ctx: rule ctx.
original_sources: `depset[File]`; the original input sources from `srcs`
required_py_files: `depset[File]`; the direct, `.py` sources for the
target that **must** be included by downstream targets. This should
only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
transitive sources collected from dependencies (the latter is only
necessary for deprecated extra actions support).
"""

py_info = PyInfoBuilder()
py_info.direct_original_sources.add(original_sources)
py_info.direct_pyc_files.add(required_pyc_files)
py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
py_info.transitive_original_sources.add(original_sources)
py_info.transitive_pyc_files.add(required_pyc_files)
py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
if f.extension == "py":
py_info.transitive_sources.add(f)
py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
for target in ctx.attr.pyi_deps:
# PyInfo may not be present e.g. cc_library rules.
if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
py_info.merge(_get_py_info(target))

deps_transitive_sources = py_info.transitive_sources.build()
py_info.transitive_sources.add(required_py_files)
Expand Down
5 changes: 5 additions & 0 deletions python/private/py_executable.bzl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
runfiles_details = runfiles_details,
main_py = main_py,
imports = imports,
original_sources = direct_sources,
required_py_files = required_py_files,
required_pyc_files = required_pyc_files,
implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
ctx,
executable,
main_py,
original_sources,
required_py_files,
required_pyc_files,
implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
ctx: The rule ctx.
executable: File; the target's executable file.
main_py: File; the main .py entry point.
original_sources: `depset[File]` the direct `.py` sources for the
target that were the original input sources.
required_py_files: `depset[File]` the direct, `.py` sources for the
target that **must** be included by downstream targets. This should
only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

py_info, deps_transitive_sources, builtin_py_info = create_py_info(
ctx,
original_sources = original_sources,
required_py_files = required_py_files,
required_pyc_files = required_pyc_files,
implicit_pyc_files = implicit_pyc_files,
Expand Down
135 changes: 133 additions & 2 deletions python/private/py_info.bzl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
direct_pyc_files = depset(),
transitive_pyc_files = depset(),
transitive_implicit_pyc_files = depset(),
transitive_implicit_pyc_source_files = depset()):
transitive_implicit_pyc_source_files = depset(),
direct_original_sources = depset(),
transitive_original_sources = depset(),
direct_pyi_files = depset(),
transitive_pyi_files = depset()):
_check_arg_type("transitive_sources", "depset", transitive_sources)

# Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

_check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
_check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

_check_arg_type("direct_original_sources", "depset", direct_original_sources)
_check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

_check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
_check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
return {
"direct_original_sources": direct_original_sources,
"direct_pyc_files": direct_pyc_files,
"direct_pyi_files": direct_pyi_files,
"has_py2_only_sources": has_py2_only_sources,
"has_py3_only_sources": has_py2_only_sources,
"imports": imports,
"transitive_implicit_pyc_files": transitive_implicit_pyc_files,
"transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
"transitive_original_sources": transitive_original_sources,
"transitive_pyc_files": transitive_pyc_files,
"transitive_pyi_files": transitive_pyi_files,
"transitive_sources": transitive_sources,
"uses_shared_libraries": uses_shared_libraries,
}
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
doc = "Encapsulates information provided by the Python rules.",
init = _PyInfo_init,
fields = {
"direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
"direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
"direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
"has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
"transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
"transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
"transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
"transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
_uses_shared_libraries = [False],
build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
direct_original_sources = builders.DepsetBuilder(),
direct_pyc_files = builders.DepsetBuilder(),
direct_pyi_files = builders.DepsetBuilder(),
get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
transitive_implicit_pyc_files = builders.DepsetBuilder(),
transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
transitive_original_sources = builders.DepsetBuilder(),
transitive_pyc_files = builders.DepsetBuilder(),
transitive_pyi_files = builders.DepsetBuilder(),
transitive_sources = builders.DepsetBuilder(),
)
return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
"""Merge other PyInfos into this PyInfo.

Args:
self: implicitly added.
*infos: {type}`PyInfo` objects to merge in, but only merge in their
information into this object's transitive fields.
direct: {type}`list[PyInfo]` objects to merge in, but also merge their
direct fields into this object's direct fields.

Returns:
{type}`PyInfoBuilder` the current object
"""
return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
"""Merge other PyInfos into this PyInfo.

Args:
self: implicitly added.
transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
their information into this object's transitive fields.
direct: {type}`list[PyInfo]` objects to merge in, but also merge their
direct fields into this object's direct fields.

Returns:
{type}`PyInfoBuilder` the current object
"""
for info in direct:
# BuiltinPyInfo doesn't have this field
if hasattr(info, "direct_pyc_files"):
self.direct_original_sources.add(info.direct_original_sources)
self.direct_pyc_files.add(info.direct_pyc_files)
self.direct_pyi_files.add(info.direct_pyi_files)

for info in direct + transitive:
self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
if hasattr(info, "transitive_pyc_files"):
self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
self.transitive_original_sources.add(info.transitive_original_sources)
self.transitive_pyc_files.add(info.transitive_pyc_files)
self.transitive_pyi_files.add(info.transitive_pyi_files)

return self

def _PyInfoBuilder_merge_target(self, target):
"""Merge a target's Python information in this object.

Args:
self: implicitly added.
target: {type}`Target` targets that provide PyInfo, or other relevant
providers, will be merged into this object. If a target doesn't provide
any relevant providers, it is ignored.

Returns:
{type}`PyInfoBuilder` the current object.
"""
if PyInfo in target:
self.merge(target[PyInfo])
elif BuiltinPyInfo != None and BuiltinPyInfo in target:
self.merge(target[BuiltinPyInfo])
return self

def _PyInfoBuilder_merge_targets(self, targets):
"""Merge multiple targets into this object.

Args:
self: implicitly added.
targets: {type}`list[Target]`
targets that provide PyInfo, or other relevant
providers, will be merged into this object. If a target doesn't provide
any relevant providers, it is ignored.

Returns:
{type}`PyInfoBuilder` the current object.
"""
for t in targets:
self.merge_target(t)
return self

def _PyInfoBuilder_build(self):
if config.enable_pystar:
kwargs = dict(
direct_original_sources = self.direct_original_sources.build(),
direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
transitive_original_sources = self.transitive_original_sources.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
transitive_pyi_files = self.transitive_pyi_files.build(),
)
else:
kwargs = {}
Expand Down
1 change: 1 addition & 0 deletions python/private/py_library.bzl
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
cc_info = semantics.get_cc_info_for_library(ctx)
py_info, deps_transitive_sources, builtins_py_info = create_py_info(
ctx,
original_sources = direct_sources,
required_py_files = required_py_files,
required_pyc_files = required_pyc_files,
implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
feat: add pyi attributes/fields, original source fields#2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

aignas
merged 5 commits into

from

Dec 31, 2024
 Merged


feat: add pyi attributes/fields, original source fields
#2538
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


5 commits


Select commit
Hold shift + click to select a range
d4fcb41

feat: add PyInfo fields for storing original sources of a target


rickeylev Nov 28, 2024
18a64db

feat: add pyi_srcs, pyi_deps attributes


rickeylev Dec 28, 2024
b5f9151

update changelog


rickeylev Dec 29, 2024
6fef69c

run precommit


rickeylev Dec 29, 2024
bd9023e

fix test


rickeylev Dec 29, 2024

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


7 changes: 7 additions & 0 deletions
CHANGELOG.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -97,6 +97,13 @@ Unreleased changes template.
 * 3.11.11
 * 3.12.8
 * 3.13.1
* (rules) Attributes for type definition files (`.pyi` files) and type-checking
 only dependencies added. See {obj}`py_library.pyi_srcs` and
 `py_library.pyi_deps` (and the same named attributes for `py_binary` and
 `py_test`).
* (providers) {obj}`PyInfo` has new fields to aid static analysis tools:
 {obj}`direct_original_sources`, {obj}`direct_pyi_files`,
 {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`.

[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206

Expand Down

29 changes: 29 additions & 0 deletions
python/private/attributes.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -375,6 +375,35 @@ in the resulting output or not. Valid values are:
* `omit_source`: Don't include the original py source.
""",
 ),
 "pyi_deps": attr.label_list(
 doc = """
Dependencies providing type definitions the library needs.

These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
These are build-time only dependencies and not included as part of a runnable
program (packaging rules may include them, however).
@aignasaignasDec 31, 2024

Copy link


Copy Markdown

Collaborator
Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
  1. What happens if pyi sources propagate through deps vs pyi_deps.
  2. Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 providers = [
 [PyInfo],
 [CcInfo],
 ] + _MaybeBuiltinPyInfo,
 ),
 "pyi_srcs": attr.label_list(
 doc = """
Type definition files for the library.

These are typically `.pyi` files, but other file types for type-checker specific
formats are allowed. These files are build-time only dependencies and not included
as part of a runnable program (packaging rules may include them, however).

:::{versionadded} VERSION_NEXT_FEATURE
:::
""",
 allow_files = True,
 ),
 # Required attribute, but details vary by rule.
 # Use create_srcs_attr to create one.
 "srcs": None,
Expand Down

11 changes: 10 additions & 1 deletion
python/private/common.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()):
def create_py_info(
 ctx,
 *,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -417,6 +418,7 @@ def create_py_info(

 Args:
 ctx: rule ctx.
 original_sources: `depset[File]`; the original input sources from `srcs`
 required_py_files: `depset[File]`; the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand All@@ -435,10 +437,13 @@ def create_py_info(
 transitive sources collected from dependencies (the latter is only
 necessary for deprecated extra actions support).
 """

 py_info = PyInfoBuilder()
 py_info.direct_original_sources.add(original_sources)
 py_info.direct_pyc_files.add(required_pyc_files)
 py_info.direct_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_original_sources.add(original_sources)
 py_info.transitive_pyc_files.add(required_pyc_files)
 py_info.transitive_pyi_files.add(ctx.files.pyi_srcs)
 py_info.transitive_implicit_pyc_files.add(implicit_pyc_files)
 py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files)
 py_info.imports.add(imports)
Expand All@@ -457,6 +462,10 @@ def create_py_info(
 if f.extension == "py":
 py_info.transitive_sources.add(f)
 py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f))
 for target in ctx.attr.pyi_deps:
 # PyInfo may not be present e.g. cc_library rules.
 if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target):
 py_info.merge(_get_py_info(target))

 deps_transitive_sources = py_info.transitive_sources.build()
 py_info.transitive_sources.add(required_py_files)
Expand Down

5 changes: 5 additions & 0 deletions
python/private/py_executable.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment =
 runfiles_details = runfiles_details,
 main_py = main_py,
 imports = imports,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand DownExpand Up@@ -1548,6 +1549,7 @@ def _create_providers(
 ctx,
 executable,
 main_py,
 original_sources,
 required_py_files,
 required_pyc_files,
 implicit_pyc_files,
Expand All@@ -1566,6 +1568,8 @@ def _create_providers(
 ctx: The rule ctx.
 executable: File; the target's executable file.
 main_py: File; the main .py entry point.
 original_sources: `depset[File]` the direct `.py` sources for the
 target that were the original input sources.
 required_py_files: `depset[File]` the direct, `.py` sources for the
 target that **must** be included by downstream targets. This should
 only be Python source files. It should not include pyc files.
Expand DownExpand Up@@ -1649,6 +1653,7 @@ def _create_providers(

 py_info, deps_transitive_sources, builtin_py_info = create_py_info(
 ctx,
 original_sources = original_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down

135 changes: 133 additions & 2 deletions
python/private/py_info.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -38,7 +38,11 @@ def _PyInfo_init(
 direct_pyc_files = depset(),
 transitive_pyc_files = depset(),
 transitive_implicit_pyc_files = depset(),
 transitive_implicit_pyc_source_files = depset()):
 transitive_implicit_pyc_source_files = depset(),
 direct_original_sources = depset(),
 transitive_original_sources = depset(),
 direct_pyi_files = depset(),
 transitive_pyi_files = depset()):
 _check_arg_type("transitive_sources", "depset", transitive_sources)

 # Verify it's postorder compatible, but retain is original ordering.
Expand All@@ -53,14 +57,24 @@ def _PyInfo_init(

 _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files)
 _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files)

 _check_arg_type("direct_original_sources", "depset", direct_original_sources)
 _check_arg_type("transitive_original_sources", "depset", transitive_original_sources)

 _check_arg_type("direct_pyi_files", "depset", direct_pyi_files)
 _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files)
 return {
 "direct_original_sources": direct_original_sources,
 "direct_pyc_files": direct_pyc_files,
 "direct_pyi_files": direct_pyi_files,
 "has_py2_only_sources": has_py2_only_sources,
 "has_py3_only_sources": has_py2_only_sources,
 "imports": imports,
 "transitive_implicit_pyc_files": transitive_implicit_pyc_files,
 "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files,
 "transitive_original_sources": transitive_original_sources,
 "transitive_pyc_files": transitive_pyc_files,
 "transitive_pyi_files": transitive_pyi_files,
 "transitive_sources": transitive_sources,
 "uses_shared_libraries": uses_shared_libraries,
 }
Expand All@@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider(
 doc = "Encapsulates information provided by the Python rules.",
 init = _PyInfo_init,
 fields = {
 "direct_original_sources": """
:type: depset[File]

The `.py` source files (if any) that are considered directly provided by
the target. This field is intended so that static analysis tools can recover the
original Python source files, regardless of any build settings (e.g.
precompiling), so they can analyze source code. The values are typically the
`.py` files in the `srcs` attribute (or equivalent).

::::{versionadded} 1.1.0
::::
""",
 "direct_pyc_files": """
:type: depset[File]

Expand All@@ -78,6 +104,21 @@ by the target and **must be included**.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "direct_pyi_files": """
:type: depset[File]

Type definition files (usually `.pyi` files) for the Python modules provided by
this target. Usually they describe the source files listed in
`direct_original_sources`. This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "has_py2_only_sources": """
:type: bool
Expand DownExpand Up@@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead.

::::{versionadded} 0.37.0
::::
""",
 "transitive_original_sources": """
:type: depset[File]

The transitive set of `.py` source files (if any) that are considered the
original sources for this target and its transitive dependencies. This field is
intended so that static analysis tools can recover the original Python source
files, regardless of any build settings (e.g. precompiling), so they can analyze
source code. The values are typically the `.py` files in the `srcs` attribute
(or equivalent).

This is superset of `direct_original_sources`.

::::{versionadded} 1.1.0
::::
""",
 "transitive_pyc_files": """
:type: depset[File]
Expand All@@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included.
These files usually come from, e.g., a library setting {attr}`precompile=enabled`
to forcibly enable precompiling for itself. Downstream binaries are expected
to always include these files, as the originating target expects them to exist.
""",
 "transitive_pyi_files": """
:type: depset[File]

The transitive set of type definition files (usually `.pyi` files) for the
Python modules for this target and its transitive dependencies. this target.
Usually they describe the source files listed in `transitive_original_sources`.
This field is primarily for static analysis tools.

:::{note}
This may contain implementation-specific file types specific to a particular
type checker.
:::

::::{versionadded} 1.1.0
::::
""",
 "transitive_sources": """\
:type: depset[File]
Expand DownExpand Up@@ -165,7 +237,9 @@ def PyInfoBuilder():
 _uses_shared_libraries = [False],
 build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k),
 build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k),
 direct_original_sources = builders.DepsetBuilder(),
 direct_pyc_files = builders.DepsetBuilder(),
 direct_pyi_files = builders.DepsetBuilder(),
 get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k),
 get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k),
 get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k),
Expand All@@ -182,7 +256,9 @@ def PyInfoBuilder():
 set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k),
 transitive_implicit_pyc_files = builders.DepsetBuilder(),
 transitive_implicit_pyc_source_files = builders.DepsetBuilder(),
 transitive_original_sources = builders.DepsetBuilder(),
 transitive_pyc_files = builders.DepsetBuilder(),
 transitive_pyi_files = builders.DepsetBuilder(),
 transitive_sources = builders.DepsetBuilder(),
 )
 return self
Expand DownExpand Up@@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value):
 return self

def _PyInfoBuilder_merge(self, *infos, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 *infos: {type}`PyInfo` objects to merge in, but only merge in their
 information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 return self.merge_all(list(infos), direct = direct)

def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 """Merge other PyInfos into this PyInfo.

 Args:
 self: implicitly added.
 transitive: {type}`list[PyInfo]` objects to merge in, but only merge in
 their information into this object's transitive fields.
 direct: {type}`list[PyInfo]` objects to merge in, but also merge their
 direct fields into this object's direct fields.

 Returns:
 {type}`PyInfoBuilder` the current object
 """
 for info in direct:
 # BuiltinPyInfo doesn't have this field
 if hasattr(info, "direct_pyc_files"):
 self.direct_original_sources.add(info.direct_original_sources)
 self.direct_pyc_files.add(info.direct_pyc_files)
 self.direct_pyi_files.add(info.direct_pyi_files)

 for info in direct + transitive:
 self.imports.add(info.imports)
Expand All@@ -240,29 +342,58 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []):
 if hasattr(info, "transitive_pyc_files"):
 self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files)
 self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files)
 self.transitive_original_sources.add(info.transitive_original_sources)
 self.transitive_pyc_files.add(info.transitive_pyc_files)
 self.transitive_pyi_files.add(info.transitive_pyi_files)

 return self

def _PyInfoBuilder_merge_target(self, target):
 """Merge a target's Python information in this object.

 Args:
 self: implicitly added.
 target: {type}`Target` targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 if PyInfo in target:
 self.merge(target[PyInfo])
 elif BuiltinPyInfo != None and BuiltinPyInfo in target:
 self.merge(target[BuiltinPyInfo])
 return self

def _PyInfoBuilder_merge_targets(self, targets):
 """Merge multiple targets into this object.

 Args:
 self: implicitly added.
 targets: {type}`list[Target]`
 targets that provide PyInfo, or other relevant
 providers, will be merged into this object. If a target doesn't provide
 any relevant providers, it is ignored.

 Returns:
 {type}`PyInfoBuilder` the current object.
 """
 for t in targets:
 self.merge_target(t)
 return self

def _PyInfoBuilder_build(self):
 if config.enable_pystar:
 kwargs = dict(
 direct_original_sources = self.direct_original_sources.build(),
 direct_pyc_files = self.direct_pyc_files.build(),
transitive_pyc_files = self.transitive_pyc_files.build(),
direct_pyi_files = self.direct_pyi_files.build(),
 transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(),
 transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(),
 transitive_original_sources = self.transitive_original_sources.build(),
 transitive_pyc_files = self.transitive_pyc_files.build(),
 transitive_pyi_files = self.transitive_pyi_files.build(),
 )
 else:
 kwargs = {}
Expand Down

1 change: 1 addition & 0 deletions
python/private/py_library.bzl

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics):
 cc_info = semantics.get_cc_info_for_library(ctx)
 py_info, deps_transitive_sources, builtins_py_info = create_py_info(
 ctx,
 original_sources = direct_sources,
 required_py_files = required_py_files,
 required_pyc_files = required_pyc_files,
 implicit_pyc_files = implicit_pyc_files,
Expand Down
Loading