skip annotations for version results

[arthurzam]

Add support for an ebuild to declare a skip annotation for a class of version results. This skip annotation must be declared before any non-comment non-empty line (most of the times it would be before the EAPI declaration). Empty lines between the copyright and the annotation are allowed.
The annotation line must be precise, only one class per line and every space is required.

For example, it would look like:

# Copyright 2023 Gentoo Authors# Distributed under the terms of the GNU General Public License v2# pkgcheck skip: PythonCompatUpdate# Approved by larry 2023-04-31# pkgcheck skip: VariableScope
EAPI=8

Resolves: #478

More thoughts

1. Is the format fine?
2. Before merge to master/before release, we must define a QA policy who and when can add a skip.
   As preliminary idea from me (non QA member!): error none, warning only with QA approval, info/style anyone?
3. Am I too hard with the syntax and placing?
4. How and if we should support annotation for package ("dir") results?
5. Should I support multiple class names in same line, separated with commas?
6. Should I add special syntax for placing QA member name and date for those that require approval?
7. Should I post to -dev ML when ready?

TODO:

• This should be just after EAPI line instead of before
• Should support comma separated list of classes on the same line
• Post to ML for comments
• Open Question for ML: auto approval on ebuild bump?

[codecov]

Codecov Report

Patch coverage: 31.57% and project coverage change: -0.12⚠️

Comparison is base (eb6a62d) 80.24% compared to head (b4b42bb) 80.12%.

Additional details and impacted files

@@ Coverage Diff @@## master #535 +/- ##
==========================================
- Coverage 80.24% 80.12% -0.12% 
==========================================
Files 56 56 Lines 8452 8470 +18 Branches 1595 1922 +327 ==========================================
+ Hits 6782 6787 +5 
Misses 1580 1580 - Partials 90 103 +13 

Impacted Files	Coverage Δ
src/pkgcheck/results.py	85.78% <0.00%> (-0.85%)	⬇️
src/pkgcheck/sources.py	58.14% <35.29%> (-1.70%)	⬇️

... and 7 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[radhermit]

If this is going to be supported in the discussed format, I'd highly recommend pulling the pragma values during source object iteration into packages... basically wrap actual package objects adding an attribute for the result class skip set (e.g. see how the bash parse tree source works). This change currently operates at a result level which, in theory, can result in a lot of redundant work (it's reading the ebuild file per version result and each ebuild can create lots of version results), not to mention having to special case package types.

In theory, implementing it like that would even make it possible to add the contextual information into the pipeline before the checks get run so packages that ignore all the results a check can return wouldn't even run the check.

[arthurzam]

An update based on benchmarks done by sam, mgorny and me.
The performance impact of the new implementation is negligible when done on full tree.

[ferringb]

@arthurzam is this something you still consider worth following up on?

[thesamesam]

It's probably one of the most common pkgcheck feature requests - to have a way to silence a warning. I think we want it.

[ferringb]

It's probably one of the most common pkgcheck feature requests - to have a way to silence a warning. I think we want it.

Confirming: this is basically what I laid out in #763?

[thesamesam]

Yes.