Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Force GitHub README to respect dark mode (function() { var style = document.createElement('style'); style.textContent = ' .markdown-body { color-scheme: dark light; } .markdown-body pre { background: #161b22 !important; } .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; } .markdown-body table th, .markdown-body table td { border-color: #30363d !important; } .markdown-body img { background: #0d1117; } .markdown-body blockquote { border-left-color: #8b949e; } .markdown-body hr { border-color: #30363d; } '; document.head.appendChild(style); })(); } } catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Highlight search terms from Google/DuckDuckGo/Bing referrer (function() { var ref = document.referrer; var terms = []; if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) { var url = new URL(ref); var q = url.searchParams.get('q') || url.searchParams.get('p'); if (q) { terms = q.split(/\s+/).filter(function(t) { return t.length > 2; }); } } if (terms.length === 0) return; var style = document.createElement('style'); style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }'; document.head.appendChild(style); function highlight(node) { if (node.nodeType === 3) { // text node var text = node.textContent; var found = false; terms.forEach(function(term) { var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi'); if (regex.test(text)) { found = true; var frag = document.createDocumentFragment(); var parts = text.split(regex); parts.forEach(function(part, i) { if (i % 2 === 0) { frag.appendChild(document.createTextNode(part)); } else { var span = document.createElement('span'); span.className = 'userscript-highlight'; span.textContent = part; frag.appendChild(span); } }); node.parentNode.replaceChild(frag, node); } }); } else if (node.nodeType === 1 && node.childNodes) { // element var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT']; if (!skipTags.includes(node.tagName)) { Array.from(node.childNodes).forEach(highlight); } } } highlight(document.body); // Re-highlight on dynamic content var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1 || node.nodeType === 3) highlight(node); }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Strip utm_, fbclid, gclid, etc. from all links on page (function() { var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid', 'ref', 'ref_src', 'source', 'medium', 'campaign']; function cleanUrl(url) { try { var u = new URL(url, window.location.origin); var changed = false; trackingParams.forEach(function(p) { if (u.searchParams.has(p)) { u.searchParams.delete(p); changed = true; } }); return changed ? u.toString() : url; } catch (e) { return url; } } function cleanLinks() { document.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } cleanLinks(); var observer = new MutationObserver(function(mutations) { mutations.forEach(function(m) { m.addedNodes.forEach(function(node) { if (node.nodeType === 1) { if (node.tagName === 'A') cleanLinks(); node.querySelectorAll('a[href]').forEach(function(a) { var clean = cleanUrl(a.href); if (clean !== a.href) a.href = clean; }); } }); }); }); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); } })(); (function(){ try { var __m = "youtube.com"; var __re = new RegExp('^' + "youtube\\.com" + ' fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Auto-enable theater mode on YouTube (function() { function tryTheater() { var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]'); if (btn && !btn.classList.contains('activated')) { btn.click(); } } // Try immediately tryTheater(); // Try after navigation (SPA) var lastUrl = location.href; setInterval(function() { if (location.href !== lastUrl) { lastUrl = location.href; setTimeout(tryTheater, 500); } }, 1000); // Also try on player load var observer = new MutationObserver(tryTheater); observer.observe(document.body, { childList: true, subtree: true }); })(); } } catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Remove or un-stick sticky/fixed headers that block content (function() { function unstick() { document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) { if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') { el.style.position = 'static'; el.style.top = 'auto'; el.style.zIndex = 'auto'; } }); } unstick(); var observer = new MutationObserver(unstick); observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] }); })(); } } catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); } })(); (function(){ try { var __m = "*"; var __re = new RegExp('^' + ".*" + ' fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev
, 'i'); if (__m === '*' || __re.test(location.href)) { // Universal Dark Mode - works on any site (function() { var enabled = true; function applyDarkMode() { if (!enabled) return; // Create style element if it doesn't exist var style = document.getElementById('universal-dark-mode-style'); if (!style) { style = document.createElement('style'); style.id = 'universal-dark-mode-style'; document.head.appendChild(style); } // Dark mode CSS - inverts colors but preserves images/video style.textContent = ' /* Invert everything except media */ html { filter: invert(1) hue-rotate(180deg) !important; background: #1a1a2e !important; } /* Restore images, videos, iframes, canvas */ img, video, iframe, canvas, svg, picture, [style*="background-image"] { filter: invert(1) hue-rotate(180deg) !important; } /* Preserve specific elements that should not be inverted */ .no-dark-mode, .no-dark-mode *, [data-theme="light"], [data-theme="light"], .ace_editor, .ace_editor *, .CodeMirror, .CodeMirror *, .monaco-editor, .monaco-editor *, .markdown-body pre, .markdown-body pre *, .highlight, .highlight *, pre code, pre code * { filter: none !important; } /* Fix common UI elements */ .modal, .popup, .dropdown-menu, .tooltip, .popover { filter: invert(1) hue-rotate(180deg) !important; background: #2d2d44 !important; border-color: #444 !important; } /* Scrollbars */ ::-webkit-scrollbar { background: #1a1a2e !important; } ::-webkit-scrollbar-thumb { background: #444 !important; } ::-webkit-scrollbar-thumb:hover { background: #555 !important; } /* Selection */ ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; } ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; } '; } function removeDarkMode() { var style = document.getElementById('universal-dark-mode-style'); if (style) style.remove(); } // Toggle with Alt+Shift+D document.addEventListener('keydown', function(e) { if (e.altKey && e.shiftKey && e.key === 'D') { e.preventDefault(); enabled = !enabled; if (enabled) { applyDarkMode(); console.log('[Universal Dark Mode] Enabled'); } else { removeDarkMode(); console.log('[Universal Dark Mode] Disabled'); } } }); // Apply on load applyDarkMode(); // Re-apply on dynamic content var observer = new MutationObserver(function(mutations) { if (enabled && !document.getElementById('universal-dark-mode-style')) { applyDarkMode(); } }); observer.observe(document.head, { childList: true }); console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle'); })(); } } catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); } })(); })(); fix(pypi): return yank reason from SimpleAPI HTML by aignas · Pull Request #3656 · bazel-contrib/rules_python · GitHub
Skip to content

fix(pypi): return yank reason from SimpleAPI HTML - #3656

Merged
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked
Mar 14, 2026
Merged

fix(pypi): return yank reason from SimpleAPI HTML#3656
rickeylev merged 1 commit into
bazel-contrib:mainfrom
aignas:aignas.fix.parse-simpleapi-yanked

Conversation

@aignas

@aignasaignas commented Mar 9, 2026

Copy link
Copy Markdown
Collaborator

This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. With this the yank reason will
be present in the logs, which should be a usability improvement.

The yanked attribute will be None if the package is not yanked and a
string value if it is yanked. An empty string value should be treated
as "yanked with no reason provided".

This implementation assumes that we have HTML escaped sequences as tag
values. It also un-escapes them when returning the strings.

The possibilities that it gives us are, but out of scope for this PR:

  • Use the data-requires-python to potentially discard any Python
    packages that are unsupported in the select_whl function.
  • Also retrieve the provenance attributes if they are supported.

Work towards #260.
Work towards #2731.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the PyPI Simple API HTML parser to accurately process package metadata, particularly addressing the data-yanked attribute. By correctly extracting and unescaping the yank reason, it resolves a previous misinterpretation of yanked packages and provides a more robust foundation for package management. This also paves the way for leveraging other HTML attributes, such as data-requires-python, for advanced package selection.

Highlights

  • Enhanced HTML Parsing: Implemented sophisticated parsing logic for PyPI Simple API HTML <a> tags to accurately extract attributes and their values.
  • Corrected Yank Reason Handling: Correctly extracts and returns the data-yanked reason, resolving an issue where its mere presence was misinterpreted as a yanked package.
  • New Utility Functions: Introduced _parse_attrs and _unescape_pypi_html helper functions for robust attribute extraction and HTML entity decoding.
  • Data Structure Update: Modified the yanked field in the returned package struct to store the string reason instead of a boolean flag.
  • Future Capabilities: Enabled future functionality to utilize data-requires-python for advanced package filtering.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • python/private/pypi/parse_simpleapi_html.bzl
    • Refactored HTML parsing logic to correctly extract attributes from <a> tags.
    • Introduced _parse_attrs function to parse HTML attributes.
    • Introduced _unescape_pypi_html function to decode HTML entities in attribute values.
    • Updated yanked field to store the yank reason string instead of a boolean.
  • tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl
    • Updated test cases to reflect the change in yanked field from boolean to string.
    • Added new test cases for data-yanked with and without a reason, including HTML escaped characters.
    • Updated data-requires-python values to use HTML escaped characters in tests.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

FeatureCommandDescription
Code Review/gemini reviewPerforms a code review for the current pull request in its current state.
Pull Request Summary/gemini summaryProvides a summary of the current pull request in its current state.
Comment@gemini-code-assistResponds in comments when explicitly tagged, both in pull request comments and review comments.
Help/gemini helpDisplays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assistgemini-code-assistBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Simple API HTML parser to correctly handle yanked packages and their reasons. However, a critical security vulnerability has been identified: the new custom HTML attribute parser fails to detect the data-yanked attribute when it is present without an explicit value. This flaw leads to a security regression, as yanked packages (often due to critical bugs or security vulnerabilities) are not recognized, potentially allowing Bazel to download and use compromised versions. Additionally, the _parse_attrs function has significant issues with its loop construct and general handling of valueless attributes, which could lead to incorrect behavior. The attribute parser must be updated to correctly handle boolean attributes and ensure data-yanked always signals a yanked status.

Comment threadpython/private/pypi/parse_simpleapi_html.bzl Outdated
@aignas
aignasforce-pushed the aignas.fix.parse-simpleapi-yanked branch from 9ad8c68 to 76f521aCompareMarch 10, 2026 10:06
This makes the logic in the parser a little bit more sophisticated, but
we also start handling the yank reason. This fixes the issue where the
`data-yank` presence but no value would be interpreted as a yanked
package. With this it should start working.
This implementation assumes that we have HTML escaped sequences as tag
values. It also unescapes them when returning the strings.
The posibilities that it gives us are:
- Use the `data-requires-python` to potentially discard any Python
packages that are unsupported in the `select_whl` function.
Work towards bazel-contrib#260.
Work towards bazel-contrib#2731.
@aignas

Copy link
Copy Markdown
CollaboratorAuthor

OK, this is done. PTAL.

@rickeylev
rickeylev added this pull request to the merge queueMar 14, 2026
Merged via the queue into bazel-contrib:main with commit 798df3fMar 14, 2026
4 checks passed
@aignas
aignas deleted the aignas.fix.parse-simpleapi-yanked branch March 14, 2026 08:49
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@aignas@rickeylev