Python warnings are not reported by CI

[jacobperron]

Neither colcon test nor Jenkins report warnings coming from Python code. For example, deprecated API was recently introduced in ros2/ros2cli#463 but CI passes without reported the warnings. I would expect an unstable build so that we notice things like deprecated Python APIs and can address them.

Here is an example build testing ros2node:

It is unstable for a different reason. The point is there are 61 warnings visible in the console, but they are not reported by Jenkins: https://ci.ros2.org/job/ci_linux/9731/consoleFull#console-section-223

Specifically for tests, I tried passing a pytest argument to treat warnings as errors (#416), but this results in a failed build, instead of unstable, which I think is not desired.

Related colcon PR to report pytest warnings: colcon/colcon-core#236

[hidmic]

@jacobperron were you able to make Jenkins pick up those warnings?

[jacobperron]

@hidmic No, I didn't look into it too deeply.

[hidmic]

Current aim is to try and figure out whether the existing Jenkins plugins can be configured to pick up python warnings in stderr (e.g. using a regex) or if a custom plugin will be required.

[nuclearsandwich]

I did some checking and it doesn't appear that a python warning parser exists as part of the Jenkins analysis-model nor is there a separate plugin which provides one. I did manage to write a regexp-based parser to extract Python warnings from Pytest warnings summaries from colcon test stderr logs.

#447 is a PR which adds a parser for the warnings as they appear in our colcon logs.

[dirk-thomas]

Neither colcon test nor Jenkins report warnings coming from Python code.

The pytest warnings are being redirected to stderr since colcon/colcon-core#338. This is matching the behavior of e.g. C++ compiler warnings.

It is up to Jenkins to extract those and change the build status based on it. See #509 for a draft PR doing that.