Skip to content

[BUG] Flat index benchmark shows 0.9999899999999999 recall #1171

Description

@arrayka
  1. Is it due to loss of precision during recall calculation?
  2. Or is it a real miss? If so, check other datasets.
cargo run --package diskann-benchmark --release -- run --input-file diskann-benchmark\perf_test_inputs\wikipedia-100K-flat-index.json --output-file ./target/tmp/flat-index-output.json     

######################
# Running Job 1 of 1 #
######################

              Data: target/tmp\wikipedia_cohere/wikipedia_base.bin.crop_nb_100000
         Data Type: float32
          Distance: inner_product
           Queries: target/tmp\wikipedia_cohere/wikipedia_query.bin
       Groundtruth: target/tmp\wikipedia_cohere/wikipedia-100K
                 K: 100
           Threads: 4, 8
              Reps: 1

Loading dataset...
  Loaded 100000 vectors of dimension 768
  Queries: 5000, Groundtruth: 5000x100


  K,   Avg cmps,   QPS - mean(max),             Avg Latency,           p99 Latency,               Recall,   Threads
===================================================================================================================
100,     100000,     123.0 (123.0),   32430.0us (32430.0us),   35711.0us (35711us),   0.9999899999999999,         4
100,     100000,     250.4 (250.4),   31909.2us (31909.2us),   33323.0us (33323us),   0.9999899999999999,         8

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions