Uh oh!
There was an error while loading. Please reload this page.
Add more selective provider tests - #24666
Conversation
5d1b3a9 to
9934de2CompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
9934de2 to
a1f9ec5Comparepotiuk
commented
Jun 27, 2022
Resolved @bhirsz :). Thanks for feedback. The for/else clause made my day. |
potiuk
commented
Jun 27, 2022
BTW. This one is going to help quite a bit with the elapsed time neeed for "localized" PRs to providers :) and together with #24671 it will be even more localised as any changes to provider including it's dependencies will limit tests to only this provider, they won't need the "core" tests at all. |
d459df2 to
c3bd97dComparepotiuk
commented
Jul 4, 2022
Anyone ? This should speed up the tests in most cases and adds a way how to run tests per provider (will be useful in the future when we split providers). |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
e75b3f0 to
928049cComparepotiuk
commented
Jul 6, 2022
Nice I added tests :). In the meantime I changed the provider_dependencies.json structure and the tests nicely caught it :) |
After implementing apache#24610 and few follow-up fixes, it is now easy to add more optimizations to our unit test execution in CI (and to give this capability back to our contributors). This PR adds capability of running tests for selected set of providers - not for the whole "Providers" group. You can specify `--test-type "Providers[airbyte,http]" to only run tests for the two selected providers. This is the step towards separating providers to separate repositories, but it also allows to optimize the experience of the contributors developing only single provider changes (which is vast majority of contributions). This also allows to optimize build and elapsed time needd to run tests for those PRs that only affects selected providers (again - vast majority of PRs). The CI selection of which provider tests is done now in Selective Checkcs - they are a bit smarter in just selecting the providers that has been changed, they also check if there are any other providers that depend on it (we keep automatically updated by pre-commit dependencies.json file and this file determines which files should be run.
928049c to
f04bc8bCompareAfter implementing #24610 and few follow-up fixes, it is now easy to add more optimizations to our unit test execution in CI (and to give this capability back to our contributors). This PR adds capability of running tests for selected set of providers - not for the whole "Providers" group. You can specify `--test-type "Providers[airbyte,http]" to only run tests for the two selected providers. This is the step towards separating providers to separate repositories, but it also allows to optimize the experience of the contributors developing only single provider changes (which is vast majority of contributions). This also allows to optimize build and elapsed time needd to run tests for those PRs that only affects selected providers (again - vast majority of PRs). The CI selection of which provider tests is done now in Selective Checkcs - they are a bit smarter in just selecting the providers that has been changed, they also check if there are any other providers that depend on it (we keep automatically updated by pre-commit dependencies.json file and this file determines which files should be run. (cherry picked from commit 3dedbd3)
After implementing #24610 and few follow-up fixes, it is now easy
to add more optimizations to our unit test execution in CI (and
to give this capability back to our contributors).
This PR adds capability of running tests for selected set of
providers - not for the whole "Providers" group. You can
specify `--test-type "Providers[airbyte,http]" to only run tests
for the two selected providers.
This is the step towards separating providers to separate
repositories, but it also allows to optimize the experience of
the contributors developing only single provider changes (which
is vast majority of contributions).
This also allows to optimize build and elapsed time needd to run
tests for those PRs that only affects selected providers (again -
vast majority of PRs).
The CI selection of which provider tests is done now in Selective
Checkcs - they are a bit smarter in just selecting the providers
that has been changed, they also check if there are any other
providers that depend on it (we keep automatically updated by
pre-commit dependencies.json file and this file determines
which files should be run.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragement file, named
{pr_number}.significant.rst, in newsfragments.