Skip to content

Add default exclusion of slow tests in pytest - #1831

Merged
antonwolfy merged 12 commits into
masterfrom
enable_mark_slow_pytest
May 17, 2024
Merged

Add default exclusion of slow tests in pytest#1831
antonwolfy merged 12 commits into
masterfrom
enable_mark_slow_pytest

Conversation

@vlad-perevezentsev

@vlad-perevezentsevvlad-perevezentsev commented May 17, 2024

Copy link
Copy Markdown
Contributor

This PR suggests to exclude running slow tests (marked as slow) when pytest is called by default and to register custom markers such as slow and multi_gpu.

These changes will help to avoid spontaneous hangs in internal CI since some tests are decorated by @testing.slow which is not implemented yet.
It is assumed that such tests (usually using huge data) should perform computation on multiple devices to reduce the computation load. Internal CI contains one device and running such tests causes spontaneous hangs.

To run all tests including those marked as slow (@testing.slow ) use pytest -m "not slow or slow"

$ pytest tests/third_party/cupy/math_tests/test_sumprod.py::TestSumprod
collected 29 items / 1 deselected / 28 selected tests/third_party/cupy/math_tests/test_sumprod.py ...........................s ========================= 27 passed, 1 skipped, 1 deselected in 1.96s=========================
$ pytest -m "not slow or slow" tests/third_party/cupy/math_tests/test_sumprod.py::TestSumprod
collected 29 items tests/third_party/cupy/math_tests/test_sumprod.py ............................s =========================28 passed, 1 skipped in 3.77s=========================

Also removes test skipping for dpnp.power() due to using the latest OneAPI compiler for coverage GitHub Action #1777

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • If this PR is a work in progress, are you filing the PR as a draft?

@github-actions

github-actionsBot commented May 17, 2024

Copy link
Copy Markdown
Contributor

View rendered docs @ https://intelpython.github.io/dpnp/index.html

@vlad-perevezentsev

Copy link
Copy Markdown
ContributorAuthor

about my last commit
@antonwolfy do we need to keep using -p no:warnings for addopts?
Does it make sense to add filterwarnings section and handle warnings in it?

@antonwolfyantonwolfy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @vlad-perevezentsev, the change is very helpful and makes the tests more stable.
The filterwarnings marker can be added separately.

@vlad-perevezentsev

Copy link
Copy Markdown
ContributorAuthor

Thank you @vlad-perevezentsev, the change is very helpful and makes the tests more stable. The filterwarnings marker can be added separately.

I will keep filterwarnings registration in markers for this PR cause running tests does not work without it.
I will remove it in the next PR along with removing -p no:warnings and add warning handling

@antonwolfy
antonwolfy merged commit ab5ed63 into masterMay 17, 2024
@antonwolfy
antonwolfy deleted the enable_mark_slow_pytest branch May 17, 2024 20:54
github-actionsBot added a commit that referenced this pull request May 17, 2024
* Register slow and multi_gpu marks for pytest
* Add default exclusion of slow tests in pytest
* Add @testing.slow for test_matmul_large
* Remove test skip for dpnp.power
* Remove unused import in test_sumprod.py
* Add filterwarnings to markers
* Remove -p no:warnings from pytest addopts
* Undo commit with remove -p no:warnings
---------
Co-authored-by: Anton <100830759+antonwolfy@users.noreply.github.com> ab5ed63
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@vlad-perevezentsev@antonwolfy