Benchmark Repository

This git repository is for the benchmarking of reranking and embedding APIs.

Setup

In order to setup this repo, ensure to install dependencies:

• Astral UV

uvsync

Then, you will want to download the data (queries and documents).

pythonsrc/download_data.py

Finally, you will want to specify the necessary environment variables,

cp .env.example .env
vim .env

Running the benchmark

To test the latency and throughput of the endpoint, you will want to do a sweep over QPS to explore the latencies you get at different amounts of traffic. Then, you can graph the results with graph_tests.ipynb

pythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps1--duration30--savelogs/rerank-qps-1.jsonpythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps2--duration30--savelogs/rerank-qps-2.jsonpythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps3--duration30--savelogs/rerank-qps-3.jsonpythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps4--duration15--savelogs/rerank-qps-4.jsonpythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps6--duration15--savelogs/rerank-qps-6.jsonpythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps8--duration15--savelogs/rerank-qps-8.jsonpythonsrc/benchmark.py--providerzeroentropy--taskrerank--qps12--duration15--savelogs/rerank-qps-12.jsonpythonsrc/benchmark.py--providerzeroentropy--taskembed-queries--qps10--duration30--savelogs/embed-queries-qps-10.jsonpythonsrc/benchmark.py--providerzeroentropy--taskembed-queries--qps50--duration30--savelogs/embed-queries-qps-50.jsonpythonsrc/benchmark.py--providerzeroentropy--taskembed-queries--qps100--duration30--savelogs/embed-queries-qps-100.jsonpythonsrc/benchmark.py--providerzeroentropy--taskembed-queries--qps200--duration30--savelogs/embed-queries-qps-200.jsonpythonsrc/benchmark.py--providerzeroentropy--taskembed-queries--qps300--duration30--savelogs/embed-queries-qps-300.jsonpythonsrc/benchmark.py--providerzeroentropy--taskembed-queries--qps500--duration15--savelogs/embed-queries-qps-500.json