Benchmarks / Vector search
Vector search, compared at the same recall.
Any vector index can be made fast by returning worse neighbours, so we report latency together with recall: the share of the true 10 nearest neighbours that came back. The workload is the one most vector databases publish against, VectorDBBench Cohere 1M x 768.
Recall vs latency
VectorDBBench Cohere 1M x 768, k=10
Down and to the right is better. Each blue point is one WarpSearch setting, from the fastest to an exact scan; each grey mark is a configuration a vendor published.
Matched recall
Engine by engine
For each published configuration we take our fastest setting whose recall is at least as high, and compare p99.
| Published engine and setting | Hardware | Their recall / p99 | WarpSearch recall / p99 | Result |
|---|---|---|---|---|
| pgvector IVF_FLAT | 2 cores | 0.890 / 731 ms | 0.902 / 12 ms | 61x faster |
| Amazon S3 Vectors | managed | 0.872 / 337 ms | 0.882 / 11 ms | 31x faster |
| turbopuffer | managed | 0.899 / 56.7 ms | 0.902 / 12 ms | 4.7x faster |
| Weaviate Cloud, business critical | managed | 0.991 / 179.5 ms | 1.000 / 124 ms | 1.4x faster, exact |
| Elastic Cloud, num_candidates 150 | 8 cores | 0.906 / 17.1 ms | 0.937 / 12 ms | 1.4x faster, higher recall |
| Weaviate Cloud, standard | managed | 0.991 / 146 ms | 1.000 / 124 ms | 1.2x faster, exact |
| OpenSearch 2.17, efSearch 40 | 16 cores | 0.914 / 13.2 ms | 0.937 / 12 ms | 1.1x faster, higher recall |
Published values: VectorDBBench result files (zilliztech/VectorDBBench, case Performance768D1M), dated 2023-07 to 2026-04. WarpSearch: Serverless, k=10, recall over 1,000 queries, single-client server p99, run 2026-11-13, 0 errors in about 200,000 queries. Published p99s are client-side on the vendor's own hardware.
Load
Load and index 1M vectors
Seconds until the collection is queryable. Shorter is better.
WarpSearch loaded through the Elasticsearch bulk API from a client 80 ms away over the internet, 0 failures. Others: VectorDBBench result files.
The conventions
How the industry benchmarks vector search
There is no single standard, but a few suites are what vendors publish against. We start with the one that has the most published competitor numbers.
| Suite | Question it answers | Who publishes | Status |
|---|---|---|---|
| VectorDBBench | Load time, recall, serial p99 and filtering, database level | Milvus, Qdrant, OpenSearch, Elastic, Weaviate, pgvector, S3 Vectors, turbopuffer | Cohere 1M, results above |
| tpuf-benchmark | Latency at a fixed query rate, 10M vectors, warm cache | turbopuffer | Results |
| Rally vector tracks | kNN through the Elasticsearch API, with quantization | Elastic (nightly, BBQ) | Next |
| OpenSearch Benchmark vectors | kNN through the OpenSearch API, Cohere 10M | OpenSearch project | Next |
| ANN-Benchmarks, Big ANN | Algorithm-level curves and billion-scale tracks | Libraries and research teams | Planned |
Reproduce
Dataset: OpenSearch Benchmark cohere-1m (1M x 768, inner product), the first 1,000 test queries, exact neighbours from the dataset file. Mapping: dense_vector, 768 dims, max_inner_product.
POST /vectors/_search
{ "knn": { "field": "vector", "query_vector": [ ... ], "k": 10,
"num_candidates": 1000, "ivf": { "nprobe": 256 } } } Run report 20261113T080348Z (WarpSearch Serverless).