← All articles

AI-Ready Search Starts With a More Efficient Engine

WarpSearch team · · 10 min read

Why the next generation of search needs native execution, better algorithms, cloud-native storage and a more disciplined way to use AI in engineering.

A search request used to begin with someone typing into a box. Search needed to be fast, and was often refined by the user once he saw the results. Today, most searches are done by agents using semantically close keywords, long phrases and in an automated fast loop.

An agent searches, reads the results, changes its question and searches again. A single task can produce several retrieval requests before a person sees an answer. Search engines need to handle that volume while meeting the same requirements for relevance, filtering, freshness and permissions.

That changes what we should expect from a search engine. Retrieval quality matters more than before, and the cost of each request starts to matter much more when software is doing the asking.

We think the next generation of search engines will be built around that constraint. They will need to do more useful work with less CPU, less memory and less infrastructure sitting idle; all while supporting more ways to search - like vectors and graphs.

Making the machinery underneath cheaper and easier to run is the only way forward. Having worked with huge search clusters in the last decade, we know first hand how important search clusters are; but also how annoying it is to get them tamed and optimized.

At WarpSearch, that is the engineering challenge we are taking on ourselves. Building to native execution, serious attention to retrieval research, an architecture designed for the cloud and the new economics, and AI used to shorten the distance between an optimization idea and a trustworthy measurement.

Get closer to the CPU

Search spends a lot of its time in fairly small pieces of code. Decode a block of document IDs. Intersect two sets. Calculate a score. Compare it with a threshold. Compute distances to candidate vectors. Do it again, millions of times.

In those loops, details accumulate. An extra allocation, an unpredictable branch or a cache miss can matter more than an elegant abstraction elsewhere in the system.

This is why we favor C++ for the search core. It gives us direct control over memory layout, allocation and the instructions used in critical paths. We can keep related values together, reuse buffers and specialize a loop for the data it actually processes. We can also inspect the generated machine code when a seemingly harmless change makes a benchmark worse.

Java made a sensible trade for a generation of infrastructure software. A managed runtime, mature tooling and a large ecosystem helped teams build complex systems. The JVM can produce excellent machine code, and Lucene already uses SIMD for vector calculations. The choice of language alone does not establish a performance advantage.

Our argument for C++ is about control. We want the freedom to decide how the hottest data is represented and when memory is reclaimed, without working around a managed runtime. That freedom comes with engineering responsibility. Memory safety, portability and debugging still need time and good tooling.

The language makes certain choices possible. Profiling tells us which choices are worth making.

The cheapest score is one you never calculate

Suppose a query matches a million documents and the caller wants the best ten. Scoring every match would waste work if we can prove that large groups of documents cannot beat the results already found.

That is the idea behind score-based pruning. Our C++ search core uses block-level score upper bounds with MaxScore-based scoring for eligible top-k queries. As the result heap fills, its lowest competitive score becomes a threshold. Blocks whose combined upper bound falls below that threshold can be skipped. MaxScore also separates clauses that must generate candidates from clauses that only need checking when a candidate could still compete.

Conceptual score bounds: blocks below the top-k threshold are skipped, while a competitive block is scored.

These optimizations preserve exact lexical rankings where the scorer's bounds and query semantics permit pruning, including the required ordering of ties. An exact match count or complete aggregations may still require visiting every match, even if ranking can skip some scoring work. The engine selects an execution strategy based on the request; specialized filtered unions, for example, use WAND-style candidate formation.

Profiling the scoring path helps us identify where tighter bounds, a different index layout or a different traversal strategy could eliminate more work. For the documents that still need scoring, our engine uses compressed postings, SIMD decoding and block scoring.

Our engine's default approximate vector index combines inverted-file (IVF) lists with RaBitQ codes; product quantization remains an explicit alternative. Small segments can use an exact scan instead. The engine selects IVF lists to visit, estimates distances from compact codes, then reads original vectors for exact rescoring of selected candidates.

RaBitQ's default candidate selection uses confidence intervals to keep potentially competitive values and avoid unnecessary exact reads. With multi-bit codes, a one-bit estimate can reject a value before the extra bits are evaluated. A fast seeded rotation shared by indexing and querying avoids storing a dense rotation matrix and avoids padding vector dimensions to a power of two.

Vector retrieval pipeline from selected IVF lists through RaBitQ estimates and competitive intervals to exact rescoring.

These vector intervals use empirical confidence choices, not deterministic containment guarantees. Exact rescoring gives exact scores to the candidates it reads; it does not make the overall approximate search an exact global top-k. Recall still needs to be measured, particularly as filters, probe budgets and data distributions change.

And we keep our eyes and ears open as research innovates. For example, SymphonyQG, accepted at SIGMOD 2025, explores a different approach: designing graph traversal and compressed vector distance calculations together. We currently use IVF, but research like this gives us alternatives to evaluate against our implementation at comparable recall, on our workloads and hardware.

Design for the cloud's economics

Efficiency inside a process is only part of the bill. Even an efficient query implementation can be expensive if every collection requires a dedicated machine running around the clock.

WarpSearch, being a cloud-native implementation, keeps collection data in object storage and uses local storage as a cache. Serving compute can be released when a collection is idle, so customers do not have to size a cluster for every workload. In fact, they never see a cluster; only fast search execution at all times.

That is the architectural direction behind WarpSearch. Collection data is kept in object storage, and serving compute can be released when a collection is idle. Customers do not have to size a cluster for every workload. The collection lifecycle and durability behavior are documented, including cold starts and the interval before committed writes are copied to object storage.

The hard work starts after choosing that architecture. Object storage has request latency and transfer costs. Index layout, request batching, caching and prefetching need to be considered together.

Thus, the Warp engine leverages facts such as field presence, numeric minima and maxima, term bounds and geographic bounds to prune collections and segments. Query planning can use them to prove that a segment cannot match before opening its data files. A year filter outside a segment's recorded range, for example, needs no scan of that segment. Missing or insufficient metadata means the engine keeps the segment in play; these summaries are conservative exclusion tests, not a replacement for matching.

A year filter uses segment summaries to skip two impossible segments and open data only for a possible match.

For segments that remain, a per-segment hotcache records the byte ranges needed to open them, allowing those metadata reads to be prefetched. Together, these mechanisms reduce repeated downloads and round trips that would otherwise wait on earlier reads. Thus, achieving high performance in all scenarios.

Cold and warm behavior also need separate attention. A benchmark that repeatedly queries an index already sitting in memory says little about the first request after an idle period. Keeping compute warm can be the right choice for a latency-sensitive workload, but it has a cost that belongs in the comparison. So the engine is being smart about it, and leverages query history and situational awareness to optimize its operations.

Cloud-native search should make those trade-offs explicit. It should let a quiet collection be cheap without pretending that waking it up is free. But still aim on queries that are as fast as possible also on colde data.

Use AI to run more experiments

Lower-level engineering has traditionally been expensive. Trying three implementations of a decoder, reproducing an algorithm from a paper or investigating a compiler's output could consume days before we even knew whether the idea helped.

AI makes more of those experiments practical, to the point writing C++ and Assembly are as cheap as writing Java code. It can help trace an unfamiliar code path, implement a candidate algorithm, generate awkward test inputs and propose alternative layouts for a hot structure. An engineer can investigate more possibilities before committing to one.

We think this changes the economics of building infrastructure. Some optimizations used to lose the argument because engineering time cost more than the hardware they might save. When the cost of an experiment falls, it becomes reasonable to revisit them.

We use AI to explore candidate implementations, run exhaustive benchmarks, then check their results and performance independently before accepting a change.

Lower operating costs, lower customer bills

Code written in C++; leveraging CPU architectures; squizzing any bit of performance from the hardware; Using latest algorithms; Skipping documents that cannot compete; excluding segments that cannot match; Fast native execution; keeping data in object storage and releasing idle compute reduces what we spend during and between queries.

Together, these choices let us dramatically lower operating costs. We pass those savings on to customers through lower pricing. That is the purpose of the engine work described here: make search affordable as datasets and request volumes grow, while meeting the workload's requirements for relevance, recall and latency.

We see the strongest need for these benefits in a few use cases:

These workloads put pressure on different parts of the bill; and also require great relevance controls. Serving them economically and in a way that makes searches super-effective requires both efficient query execution and infrastructure that can scale down when demand falls. We are building WarpSearch around both.

← All articles

Cookie settings

Optional tracking stays off until you choose it. Your choice applies to this browser for 180 days. Read the details.

Essential storage keeps you signed in, protects forms, and remembers this choice. It is always available.