Far too many timeout failures on the Iris repo recently.
It's a general drag on development, and has also led to things being merged with outstanding problems.
Some ideas:
I think we have reached a point where we need to revise the testing to get back to useful CI tests that "just work". It could be by dividing a "CI acceptance" test set from the "full" tests. Alternatively, we could maybe provision our own self-maintained CI server to replace Travis testing.
Splitting the tests would require us to choose what we do about a change that passes CI tests but later turns out to fail "full" tests.
A self-provided CI service would need sufficient resourcing, and must still work for all developers.
Following relevant comment : #2860 (comment)
Far too many timeout failures on the Iris repo recently.
It's a general drag on development, and has also led to things being merged with outstanding problems.
Some ideas:
I think we have reached a point where we need to revise the testing to get back to useful CI tests that "just work". It could be by dividing a "CI acceptance" test set from the "full" tests. Alternatively, we could maybe provision our own self-maintained CI server to replace Travis testing.
Splitting the tests would require us to choose what we do about a change that passes CI tests but later turns out to fail "full" tests.
A self-provided CI service would need sufficient resourcing, and must still work for all developers.
Following relevant comment : #2860 (comment)