cloudbench

A project to bench ffmpeg performances and price for the major cloud providers.

Concept

Comparing performance and price of cloud compute instances is a complex task, given the diversity of architectures offered by the cloud providers. This project will deploy machines (using Terraform) and code (using ansible) to bench the machine and gather the results in a CSV File.

Under the hoods

• A debian 10 image is used with the latest official ffmpeg (from the debian repos)
• The compute pricing are difficult to fetch automatically, hence prices are locally stored as of now (see pricing/aws/ireland.csv and pricing/gcp/netherlands.csv)

Pre-requisites

• Terraform, Ansible and jq need to be installed
• You must have a valid account on GCP, AWS and Azure (and you must be authentified through the appropriate cli tools: gcloud, az and aws)

How to use

• campaign files are json files that can easily modified to make more/less ffmpeg computing (see campaign.json)
• machine files are json files that describe the machines to be used for testing (see machines.json)

foo@bar:~$ ./bench.sh -c campaign.json -m machines.jsonfoo@bar:~$ ./gather.sh results/campaignfoo@bar:~$ cat results/campaign.csv | heada1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf25-506MHz-2019-01-22.ts;0x78;1920x1080@6000000;fast;35553a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf25-506MHz-2019-01-22.ts;0x140;1920x1080@6000000;fast;38428a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf25-506MHz-2019-01-22.ts;0x1a4;1920x1080@6000000;fast;36509a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf25-506MHz-2019-01-22.ts;0x208;1920x1080@6000000;fast;41406a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf25-506MHz-2019-01-22.ts;0x26c;1920x1080@6000000;fast;46119a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf30-546MHz-2019-01-22.ts;0x78;1920x1080@6000000;fast;41040a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf30-546MHz-2019-01-22.ts;0xdc;1920x1080@6000000;fast;45156a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf30-546MHz-2019-01-22.ts;0x140;1920x1080@6000000;fast;41275a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf30-546MHz-2019-01-22.ts;0x1a4;1920x1080@6000000;fast;37311a1.4xlarge; ,4656; ,1375; ,2933; ,2011;tnt-uhf30-546MHz-2019-01-22.ts;0x208;1920x1080@6000000;fast;43401

TODO

• Automatic price update (static for now)
• Launch tests in parallel