Speedup integration tests

[yusufraji]

Speedup integration tests with rayon.

The original split-patches-in-dir shell script used for testing has been rewritten in Rust (test-split-patches-in-dir.rs). This allows us to run things in parallel with rayon.

Closes#12

[pflanze]

PS. some numbers about the success follow--running the integration tests is now 10-20 times faster. The bottleneck are now the other parts of the test suite (especially building the code for the unit tests); but there is room for adding more test files now, should we need them, also, thanks to this optimization the memory checker is now very cheap, which means we'll be able to make it the default once we figure out what to do on macOS.

[pflanze]

On my laptop (in the VM), make test (which includes cargo test) went from 3.6 s to 0.30 s.

make test_integration went from 2.8 s to 0.12 s:

real	0m0.117s
user	0m0.189s
sys	0m0.126s

make test_integration_opt went from 2.0 s to 0.11 s:

real	0m0.113s
user	0m0.051s
sys	0m0.150s

This indicates that the bottleneck is not the speed of split_patch or diff anymore but of the kernel (file I/O).

make leak_test, which is run by GitHub CI (on a much slower machine, thus taking around 20 seconds), took 6.1 s, and now runs in 0.35 s (this again also includes cargo test).

real	0m6.119s
user	0m4.402s
sys	0m1.916s

to

real	0m0.355s
user	0m0.528s
sys	0m0.311s

This shows that the leak sanitizer costs quite a bit of time per process, and now that cost is amortized across a whole integration test set. (I.e. it is basically for free now.)

[pflanze]

Ah, CI is not faster than before because cargo +nightly rebuilds everything again; looks like GitHub updates the nightly compiler and then does not use the cache anymore. Another reason to change the leak tester to jemalloc which can be done with stable. Although they probably also update stable every 6 weeks.

[yusufraji]

PS. some numbers about the success follow--running the integration tests is now 10-20 times faster. The bottleneck are now the other parts of the test suite (especially building the code for the unit tests); but there is room for adding more test files now, should we need them, also, thanks to this optimization the memory checker is now very cheap, which means we'll be able to make it the default once we figure out what to do on macOS.

At the time of writing, AddressSanitizer isn't supported on aarch64-apple-darwin (rust-lang/rust#98473)

[pflanze]

At the time of writing, AddressSanitizer isn't supported on aarch64-apple-darwin (rust-lang/rust#98473)

Yes, and while the problem appears to be tiny (false positive reports) and apparently can be fixed locally by configuring ignores, that will be a bit of a pain. And not having to use +nightly is better, too. jemalloc does add a dependency, though, but it should be worth getting to know (jemalloc is a good allocator for parallel workloads; it won't matter here because split-patch uses regions (bumpalo) for allocation, thus hits the global memory allocator only rarely, but it will be useful in other apps).