Benchmarks / Vector search

Vector search, compared at the same recall.

Any vector index can be made fast by returning worse neighbours, so we report latency together with recall: the share of the true 10 nearest neighbours that came back. The workload is the one most vector databases publish against, VectorDBBench Cohere 1M x 768.

31xfaster p99 than Amazon S3 Vectors at the same recall
61xfaster p99 than pgvector, 4.7x faster than turbopuffer
0.88 / 11 msrecall@10 / p99, default settings
0.96 / 17 msrecall@10 / p99, nprobe 256

Recall vs latency

VectorDBBench Cohere 1M x 768, k=10

Down and to the right is better. Each blue point is one WarpSearch setting, from the fastest to an exact scan; each grey mark is a configuration a vendor published.

1 ms10 ms100 ms1 s 0.800.850.900.951.00 recall@10 p99 latency (log scale) Amazon S3 Vectors (managed) pgvector IVF_FLAT, 2 cores turbopuffer (managed) Elastic Cloud OpenSearch Weaviate Cloud WarpSearch exact scan
WarpSearch v0.4, p99Published configuration (VectorDBBench result files)

Matched recall

Engine by engine

For each published configuration we take our fastest setting whose recall is at least as high, and compare p99.

Published engine and settingHardwareTheir recall / p99WarpSearch recall / p99Result
pgvector IVF_FLAT2 cores0.890 / 731 ms0.902 / 12 ms61x faster
Amazon S3 Vectorsmanaged0.872 / 337 ms0.882 / 11 ms31x faster
turbopuffermanaged0.899 / 56.7 ms0.902 / 12 ms4.7x faster
Weaviate Cloud, business criticalmanaged0.991 / 179.5 ms1.000 / 124 ms1.4x faster, exact
Elastic Cloud, num_candidates 1508 cores0.906 / 17.1 ms0.937 / 12 ms1.4x faster, higher recall
Weaviate Cloud, standardmanaged0.991 / 146 ms1.000 / 124 ms1.2x faster, exact
OpenSearch 2.17, efSearch 4016 cores0.914 / 13.2 ms0.937 / 12 ms1.1x faster, higher recall

Published values: VectorDBBench result files (zilliztech/VectorDBBench, case Performance768D1M), dated 2023-07 to 2026-04. WarpSearch: Serverless, k=10, recall over 1,000 queries, single-client server p99, run 2026-11-13, 0 errors in about 200,000 queries. Published p99s are client-side on the vendor's own hardware.

Load

Load and index 1M vectors

Seconds until the collection is queryable. Shorter is better.

Cohere 1M x 768, load to queryableup to 7.6x faster
WarpSearch1,342 s
Qdrant Cloud1,500 s
S3 Vectors2,971 s
OpenSearch3,572 s
Weaviate3,674 s
pgvector10,248 s

WarpSearch loaded through the Elasticsearch bulk API from a client 80 ms away over the internet, 0 failures. Others: VectorDBBench result files.

The conventions

How the industry benchmarks vector search

There is no single standard, but a few suites are what vendors publish against. We start with the one that has the most published competitor numbers.

SuiteQuestion it answersWho publishesStatus
VectorDBBenchLoad time, recall, serial p99 and filtering, database levelMilvus, Qdrant, OpenSearch, Elastic, Weaviate, pgvector, S3 Vectors, turbopufferCohere 1M, results above
tpuf-benchmarkLatency at a fixed query rate, 10M vectors, warm cacheturbopufferResults
Rally vector trackskNN through the Elasticsearch API, with quantizationElastic (nightly, BBQ)Next
OpenSearch Benchmark vectorskNN through the OpenSearch API, Cohere 10MOpenSearch projectNext
ANN-Benchmarks, Big ANNAlgorithm-level curves and billion-scale tracksLibraries and research teamsPlanned

Reproduce

Dataset: OpenSearch Benchmark cohere-1m (1M x 768, inner product), the first 1,000 test queries, exact neighbours from the dataset file. Mapping: dense_vector, 768 dims, max_inner_product.

POST /vectors/_search
{ "knn": { "field": "vector", "query_vector": [ ... ], "k": 10,
           "num_candidates": 1000, "ivf": { "nprobe": 256 } } }

Run report 20261113T080348Z (WarpSearch Serverless).

Cookie settings

Optional tracking stays off until you choose it. Your choice applies to this browser for 180 days. Read the details.

Essential storage keeps you signed in, protects forms, and remembers this choice. It is always available.