Add benchmark suite

[eric-czech]

We should track method performance using a benchmark suite like @alimanfoo mentioned in https://github.com/pystatgen/sgkit/pull/36#issuecomment-658893949.

It would be ideal if this ran as a part of the build, but timings gathered that way are probably not very reliable. Apparently integration of ASV suites in builds is usually just to make sure they work: astropy/astropy#6149. Presumably we should have a separate, scheduled process that runs on dedicated resources in order to benchmark accurately.

As @hammer mentions in related-sciences/gwas-analysis#10, https://github.com/ornl-oxford/genben could also be a helpful resource (scikit-allel benchmarks).

Ideas for benchmarking:

• sgkit.utils.max_str_len (see https://github.com/pystatgen/sgkit/pull/314#discussion_r502590724)

[hammer]

Wes has a lot of thoughts on this topic, e.g. https://discuss.ossdata.org/t/pooling-efforts-on-continuous-benchmarking-cb/206 which led to https://github.com/conbench/conbench.

[eric-czech]

If our benchmarks involve single server execution, dask scheduler may be a useful dimension to add to how they are parameterized (cf. https://github.com/pystatgen/sgkit/issues/48#issuecomment-668665296).

[ravwojdyla]

Just wanna drop ann-benchmarks as an inspiration. From readme:

Doing fast searching of nearest neighbors in high dimensional spaces is an increasingly important problem, but so far there has not been a lot of empirical attempts at comparing approaches in an objective way.
This project contains some tools to benchmark various implementations of approximate nearest neighbor (ANN) search for different metrics. We have pregenerated datasets (in HDF5) formats and we also have Docker containers for each algorithm. There's a test suite that makes sure every algorithm works.

[hammer]

Dusting off an old Discourse post that contains a nice list of core operations: https://discourse.pystatgen.org/t/core-operations-in-human-gwas-workloads/41. We should probably have a benchmark for each of these core operations.

[tomwhite]

The code I used for benchmarking the GWAS workload on a cluster (using #438) is here: https://github.com/tomwhite/gwas-benchmark. These benchmarks are for running on an ad hoc basis, and involve quite a few manual steps (documented in the README).

A few questions:

• How could it be more automated?
• Where should results be recorded?
• Where should it live?

I think it would be useful to have a cutdown version that runs on a single machine rather than a cluster, but is longer running than the micro benchmarks in #458.

[hammer]

Conbench from Ursa Labs could be useful for us: https://ursalabs.org/blog/announcing-conbench. cc @arunkk09 and @LiangdeLI