gh-112301: Compiler warning management tooling by nohlson · Pull Request #121730 · python/cpython

nohlson · 2024-07-13T21:53:41Z

#112301: This is the first version of compiler warning check tooling integrated into the Ubuntu build ant test GitHub action job.

The tool parses compiler output and checks against an ignore file, which is a list of files for which warnings should be ignored. The script includes options to fail on regression, or fail on improvement (if a file is in the ignore list but the compiler emitted no warnings).

A more detailed writeup about the idea behind the tool can be found here

The current configuration in reusable-ubuntu.yml does not include flags to --fail-on-regression or --fail-on-improvement and simply writes to the log warnings that were emitted but not ignored, or file names that were unexpectedly missing warnings but will not fail the step in the job. Per discussion in the issue we can eventually enable those options.

Future improvements would include removal of duplicate warnings, warning count per file tracking, and adding GitHub Actions annotations.

Issue: Consider applying flags for warnings about potential security issues #112301

hugovk · 2024-07-14T09:28:57Z

(I updated the branch from main to fix the unrelated docs error: Error: new NEWS nits: /home/runner/work/cpython/cpython/build/NEWS:72: WARNING: py:func reference target not found: warnings.filterswarnings.)

hugovk

Looking good! Here's a few comments and questions.

hugovk · 2024-07-14T09:26:31Z

+ try:
+ json_data = json.loads(array)
+ json_objects_in_array = [entry for entry in json_data]
+ compiler_warnings.extend([entry for entry in json_objects_in_array if entry.get('kind') == 'warning'])


Please could you limit lines to 79 chars?
https://peps.python.org/pep-0008/#maximum-line-length
This file is in Tools/, I think it's okay to just run Black on it:
black --line-length 79 Tools/build/check_warnings.py

Broke this line into two

hugovk · 2024-07-14T09:33:14Z

+import sys
+from pathlib import Path
+import argparse
+import json
+import re


Let's sort 'em:
Suggested change
importsys
frompathlibimportPath
importargparse
importjson
importre
importargparse
importjson
importre
importsys
frompathlibimportPath

Sorted imports

hugovk · 2024-07-14T10:04:38Z

+ for key in ['caret', 'start', 'end']:
+ if key in location:
+ file = location[key]['file']
+ file = file.lstrip('./') # Remove leading curdir if present


Suggested change
file=file.lstrip('./') # Remove leading curdir if present
file=file.lstrip('./') # Remove leading current directory if present

Changed comment

hugovk · 2024-07-14T10:04:46Z

+ for key in ['caret', 'start', 'end']:
+ if key in location:
+ file = location[key]['file']
+ file = file.lstrip('./') # Remove leading curdir if present


Suggested change
file=file.lstrip('./') # Remove leading curdir if present
file=file.lstrip('./') # Remove leading current directory if present

Changed comment

sethmlarson

Thanks @hugovk for reviewing too, I have some more comments:

nohlson · 2024-07-17T03:41:06Z

I have made the requested changes; please review again

bedevere-app · 2024-07-17T03:41:11Z

Thanks for making the requested changes!

: please review the changes made to this pull request.

sethmlarson

Few more comments, otherwise looks good for the first PR :)

hugovk · 2024-07-26T06:35:03Z

+import sys
+from pathlib import Path
+
+def extract_warnings_from_compiler_output(compiler_output: str) -> list[dict]:


This looks almost good to merge, just some style nits :)
Technically we're outside the stdlib so strictly speaking PEP 8 doesn't apply, but let's follow it anyway. For example, please could you add extra newlines before these defs and wrap things to 79 chars?
The easiest way is to run Black (which we don't run on the stdlib but again we're outside the stdlib :)
black --line-length 79 Tools/build/check_warnings.py
Then some comments and long strings will need manual wrapping.
Suggested change
defextract_warnings_from_compiler_output(compiler_output: str) ->list[dict]:
defextract_warnings_from_compiler_output(compiler_output: str) ->list[dict]:
Also, please don't force push in this repo, it makes it easier to review new commits, and we squash merge everything at the end anyway. I think force-pushing is sometimes to blame for pinging lots of unrelated CODEOWNERS too.
In order to keep the commit history intact, please avoid squashing or amending history and then force-pushing to the PR. Reviewers often want to look at individual commits.
https://devguide.python.org/getting-started/pull-request-lifecycle/#quick-guide
Thanks!

Updated to adhere to PEP8.
After a rebase there were several commits that came from branches from different forks. Not really sure how that happened but I had to revert to my last commit and force push to get rid of them.

Thank you! I just pushed a commit to also wrap comments and strings to 79 chars as well.

hugovk

Thanks!

picnixz · 2024-07-27T09:40:58Z

@@ -0,0 +1,195 @@
+#!/usr/bin/env python3


Do we need this one?

I meant to comment this too: no, I don't think we do because the file does not have the executable bit and we're running it via python Tools/build/check_warnings.py

hugovk · 2024-07-30T06:44:30Z

@picnixz Your review comments came after I'd enabled auto-merge, sorry about that!

@nohlson Hi! @corona10 noticed that with this change, the build output is no longer being output to the logs, because we're redirecting it to a file. It can sometimes be useful to see this output.

Before: https://github.com/python/cpython/actions/runs/10122181963/job/27993976874
After: https://github.com/python/cpython/actions/runs/10122668622/job/27995097214

@zware suggested a command like this:

set -o pipefail; make -j4 2>&1| tee compiler_output.txt

Please could you open a PR to try this out? Thank you!

nohlson requested review from ezio-melotti and hugovk as code owners July 13, 2024 21:53

bedevere-appBot mentioned this pull request Jul 13, 2024
Consider applying flags for warnings about potential security issues #112301
Closed

bedevere-appBot added the awaiting review label Jul 13, 2024

hugovk reviewed Jul 14, 2024
View reviewed changes

sethmlarson reviewed Jul 15, 2024
View reviewed changes

Comment threadTools/build/check_warnings.py
Comment threadTools/build/check_warnings.py
Comment threadTools/build/check_warnings.py Outdated

bedevere-appBot added awaiting change review and removed awaiting review labels Jul 17, 2024

nohlsonand others added 12 commits July 23, 2024 03:14

Create simple warning check tool and add to ubuntu build and test job
5148727

Add flags to check warnings script to fail on regression or improvement
3bd5a10

Remove redundant comment
6813003

Rename warnigore file to warningignore
ab4d754

Use regex to extract json arrays
615d228

Trim whitespace
fc0a60b

Test on github unexpected improvement
7793d80

Add config for improve fail check
99715d2

Revert to prod check warning state
bc44ec2

📜🤖 Added by blurb_it.
e2ca75f

Refactor creating set of files with warnings to a dedicated function
bd1634e

Move cflags configure option to top level build configuration
b07b1d6

nohlson force-pushed the warning-management-tooling branch from 767308a to b07b1d6Compare July 23, 2024 08:18

nohlson added 2 commits July 23, 2024 03:20

Add json diagnostics to ubuntu configuration as first argument
e1954a5

Add newline to news
5935143

sethmlarson reviewed Jul 23, 2024
View reviewed changes

Comment threadTools/build/check_warnings.py Outdated
Comment threadTools/build/check_warnings.py

nohlson mentioned this pull request Jul 24, 2024
gh-112301: Add macOS warning tracking tooling #122211
Merged

sethmlarson approved these changes Jul 24, 2024
View reviewed changes

bedevere-appBot removed the awaiting change review label Jul 24, 2024

sethmlarson requested review from ezio-melotti and removed request for 1st1, JelleZijlstra, abalkin, ambv, asvetlov, ericsnowcurrently, ericvsmith, iritkatriel, markshannon, methane, pablogsal, pganssle, rhettinger and willingc July 25, 2024 04:13

hugovk reviewed Jul 26, 2024
View reviewed changes

nohlsonand others added 3 commits July 26, 2024 17:40

Enforce PEP8 on check_warnigns
0c0fc31

Add period to news
e568d6c

Wrap comments and strings to 79 chars
9f6281e

hugovk approved these changes Jul 27, 2024
View reviewed changes

bedevere-appBot added awaiting merge and removed awaiting change review labels Jul 27, 2024

hugovk enabled auto-merge (squash) July 27, 2024 09:35

picnixz reviewed Jul 27, 2024
View reviewed changes

hugovk merged commit 8ac5565 into python:mainJul 27, 2024

bedevere-appBot removed the awaiting merge label Jul 27, 2024

This was referenced Jul 30, 2024
gh-112301: Add argument aliases and tee compiler output for check warnings #122465
Merged
gh-112301: Add -Wformat=2 compiler option to NODIST #122474
Closed

nohlson mentioned this pull request Aug 7, 2024
Add compiler warning tracking tooling failure remediation guidance python/devguide#1364
Merged

-importsys
-frompathlibimportPath
-importargparse
-importjson
-importre
+importargparse
+importjson
+importre
+importsys
+frompathlibimportPath

	file=file.lstrip('./') # Remove leading curdir if present
	file=file.lstrip('./') # Remove leading current directory if present

	defextract_warnings_from_compiler_output(compiler_output: str) ->list[dict]:

	defextract_warnings_from_compiler_output(compiler_output: str) ->list[dict]:

Uh oh!

Uh oh!

Conversation

nohlson commented Jul 13, 2024• edited by bedevere-app BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugovk commented Jul 14, 2024

Uh oh!

hugovk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sethmlarson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nohlson commented Jul 17, 2024

Uh oh!

bedevere-appBot commented Jul 17, 2024

Uh oh!

sethmlarson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hugovkJul 26, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hugovk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hugovk commented Jul 30, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nohlson commented Jul 13, 2024•
edited by bedevere-app Bot
Loading

hugovkJul 26, 2024•
edited
Loading

hugovk commented Jul 30, 2024•
edited
Loading