Cleanup benchmarks

[phadej]

• Make cabal.benchmarks.project without aeson library. This would allow to not recompile criterion all the time.
• remove old python scripts
• rather make a script which would allow comparing benchmarks using https://hackage.haskell.org/package/criterion-compare
  • that script could run all benchmarks and collect their results, we have plenty of benchmarks for special cases
• Add more examples than just Twitter. for example
  • GitHub issues listing
  • https://openweathermap.org/api
  • Some NPM package's package.json and or Cabal's plan.json

[bergmark]

@phadej just checking the status here.

• First point is done,
• The python scripts are still in the code base but not used so they could just be deleted?
• benchmark compare is done.
• We still want more examples

[phadej]

@bergmark I think you are mostly correct. Benchmark compare setup is done, but there is just few benchmarks of all existing ones.

That is a bit hard to explain, so I'm not sure whether this is really good first issue. In particular I want to go through the rest of benchmarks and check whether it even make sense to have them (that is also cleanup).

[bergmark]

alright!

[ethercrow]

@phadej @bergmark Hello, this ticket looks stalled and also blocking other promising optimization tickets. Is there something an outsider could help with?

[phadej]

@ethercrow If you try to run aeson own benchmarks, you should see that they are crap. All actionable ideas to improve them, and also actions are welcome.

In my previous comment I tried to say, that the task of looking through them is not a good first issue. The person have to be familiar with aeson to be able to judge whether an old benchmark is worth keeping, for example.

Adding new good benchmarks (i.e. which results can be extrapolated to real world usage) would be very nice as well. That's very hard though.

This ticket will be solved when we can be proud about aeson's benchmark suite.

[ethercrow]

@phadej what do you think about #803 ? I didn't find any issues about possibly replacing criterion with gauge, so I took a stab.