MNT add CI jobs to run integration tests for downstream projects: loky, joblib, distributed

[pierreglaser]

This PR configures cron-jobs that build and launch the test suite of downtream projects that rely on cloudpickle. The main purpose is to find out quickly if we introduce breaking changes.

Ideally we would rather test the stable (PyPI) rather than the latest (master branch) version, but each time, a project-specific issue was making it at least clumsy, at worst impossible. (I can detail upon request).

[codecov-io]

Codecov Report

Merging #236 into master will not change coverage.
The diff coverage is n/a.

@@ Coverage Diff @@## master #236 +/- ##
=======================================
Coverage 85.12% 85.12% =======================================
Files 1 1 Lines 585 585 Branches 117 117 =======================================
Hits 498 498 Misses 63 63 Partials 24 24

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca8be49...5b468cd. Read the comment docs.

[ogrisel]

Thanks for tackling this. Here are some comments.

[pierreglaser]

Mmh. So the good news is that there does not seem to be any syntax error. Also, loky and joblib tests passed. However, 3 distributed tests failed, but travis is still green...

[pierreglaser]

OK so apparentlty openssl is using using SSLv3, whereas distributed would like us to use TLSv1.2 at least. @ogrisel is this worth the hassle? Should we switch to an anaconda distribution or skip the tests? @lesteve are you familiar with any of this? If yes, any help would be much appreciated :)

[lesteve]

@lesteve are you familiar with any of this? If yes, any help would be much appreciated :)

No suggestion off the top of my head, sorry ...

[ogrisel]

Instead of running those integration tests as cron jobs, I am thinking we could run them conditionally whenever the commit message has a special flag. For instance, for the joblib entry:

commit_message =~ /(ci-downstream|ci-joblib)/

https://docs.travis-ci.com/user/conditions-v1

WDYT?

[pierreglaser]

This sounds great to tackle issues early. But it will be harder for us to spot a breaking change if we miss it initially. We would at least have to push empty commits with ci-downstream flags before each new PyPI release.

[ogrisel]

We would at least have to push empty commits with ci-downstream flags before each new PyPI release.

Yes but we can do that in a PR that we do not merge to avoid polluting master with empty commits.

[ogrisel]

But it will be harder for us to spot a breaking change if we miss it initially.

So be it. It's a tradeoff between long CI jobs, unwanted noisy notifications (in case the downstream master branches are failing randomly for causes unrelated to cloudpickle) and thinking about doing manual triggers prior to making a cloudpickle release.

[pierreglaser]

OK. I'll change the trigger and make a few test commits inside the PR with the appropriate flags.

[ogrisel]

Actually, changing my mind on the commit message tagging syntax:

[ogrisel]

It seems to work. Can you just try with [ci downstream] to make sure that it will trigger all of them at once?

[ogrisel]

Did you cancel the [ci distributed] build on purpose?

[pierreglaser]

Did you cancel the [ci distributed] build on purpose?

Nope, actually I thought you did. Might be because of auto cancelling builds when more recent commits are pushed?

[pierreglaser]

Besides that the filter seems to work indeed.

[ogrisel]

Good, let's wait for the final build to complete.

[ogrisel]

Merged. Thanks @pierreglaser!