feat: add pyi attributes/fields, original source fields by rickeylev · Pull Request #2538 · bazel-contrib/rules_python

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev · 2024-12-27T00:31:08Z

This adds attributes and fields of use to static analysis.

For type definition files (usually .pyi files), the pyi_srcs and pyi_deps
fields are added to the rules. They end up in the PyInfo fields direct_pyi_files
and transitive_pyi_files.

So that static analysis tools can retain access to a target's Python source files,
even if precompiling is enabled, direct_original_sources and
transitive_original_sources fields are added to PyInfo.

Work towards #2537, #296

aignas

LGTM. Gazelle support can come in a later PR.

FYI @ewianda, @dougthor42.

aignas · 2024-12-31T06:04:20Z

+
+These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`.
+These are build-time only dependencies and not included as part of a runnable
+program (packaging rules may include them, however).


Gazelle has a feature that populates the deps or data field for the pyi packages that one may have in the requirement file. I am not sure how big of a lift it is to accommodate for that but it raises some questions.
What happens if pyi sources propagate through deps vs pyi_deps.
Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

rickeylev · 2024-12-31T23:25:27Z

What happens if pyi sources propagate through deps vs pyi_deps.

The deps attribute will try to include things into the final output for runtime. The pyi_deps attribute won't.

Given a target with only pyi files (i.e it only has things for build time, nothing for runtime), it doesn't matter much if something is put into deps or pyi_deps.

Given a target with e.g. both py and pyi files (i.e it has things for build time and things for runtime), putting it in pyi_deps means none of it will end up in the final output. This is useful if a source file has something guarded by typing.TYPE_CHECKING (the type checker needs the additional dependencies, but the runtime doesn't). Putting it in the deps attribute means the py files will be included in the final output (like normal), and the pyi files won't.

Do I understand correctly that the pyi_deps will not be fetched if none of the actions consumes them?

It'll be fetched, but it won't cause any actions to actually run. The analysis phase will need to resolve the target, but (assuming there is no type checker using the pyi files) none of the files/actions will be executed.

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538 CMK-20994 Change-Id: I22b07e6d9711bfe50670c1498a886efb15f020ac

Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. #2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>

#2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor <dougthor42@users.noreply.github.com>

rickeylev force-pushed the feat.orig.sources branch 4 times, most recently from 150941c to 4a4475aCompare December 27, 2024 07:07

rickeylev marked this pull request as ready for review December 27, 2024 07:22

rickeylev requested a review from aignas as a code owner December 27, 2024 07:22

feat: add PyInfo fields for storing original sources of a target
d4fcb41

rickeylev force-pushed the feat.orig.sources branch from 4a4475a to d4fcb41Compare December 27, 2024 07:58

feat: add pyi_srcs, pyi_deps attributes
18a64db

rickeylev changed the title ~~feat: add PyInfo fields for storing original sources of a target~~feat: add PyInfo fields for original sources and pyi of a targetDec 29, 2024

update changelog
b5f9151

rickeylev changed the title ~~feat: add PyInfo fields for original sources and pyi of a target~~feat: add pyi attributes/fields, original source fieldsDec 29, 2024

rickeylev added 2 commits December 28, 2024 21:46

run precommit
6fef69c

fix test
bd9023e

aignas approved these changes Dec 31, 2024
View reviewed changes

aignas added this pull request to the merge queue Dec 31, 2024

Merged via the queue into bazel-contrib:main with commit c8346f9Dec 31, 2024

rickeylev deleted the feat.orig.sources branch December 31, 2024 23:25

rickeylev mentioned this pull request Jan 2, 2025
Strict rules that check dependencies and type annotations #296
Open

Synss added a commit to Synss/rules_mypy.orig that referenced this pull request Feb 12, 2025
Add support for pyi_srcs / direct_pyi_files
5da1cd8
Introduced in rules_python 1.0. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Synss mentioned this pull request Feb 12, 2025
Add support for pyi_srcs / direct_pyi_files bazel-contrib/rules_mypy#68
Merged

Synss added a commit to Synss/rules_mypy.orig that referenced this pull request Feb 13, 2025
Add support for pyi_srcs / direct_pyi_files
7cd9123
Introduced in rules_python 1.1. See: * https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 * bazel-contrib/rules_python#2538

Synss mentioned this pull request Feb 13, 2025
feat: Package pyi files in wheel #2609
Merged

amartani mentioned this pull request Jun 20, 2025
feat(gazelle): Add type-checking only dependencies to pyi_deps #3014
Merged