For AI agents: this page is also available as Markdown at /docs/hybrid-search.md. Every docs page is: add .md to its URL. llms.txt lists them all.

Hybrid search

Keyword search and vector search fail in opposite ways. Keywords miss paraphrases and synonyms, and vectors miss exact identifiers, rare names and part numbers. Hybrid search runs both and merges the results, so a query like "waterproof jacket for hiking" benefits from meaning, and "XJ-4421" still finds the exact product.

WarpSearch gives you two ways to do it:

• the native fusion operation, which ranks each source on its own and merges the lists with reciprocal rank fusion (RRF) or weighted linear fusion, and
• the Elasticsearch and OpenSearch style, where a knn section sits beside a text query and the scores are summed.

#Native fusion

Coming soon: the fusion operation (RRF and weighted linear fusion) is implemented in the search engine but the public API does not accept it yet. Today, build hybrid queries with a knn section beside a query (Hybrid with the Elasticsearch-compatible API), where scores are summed, or run the two searches and merge the lists in your application. The details below describe the behavior you can expect.

A fusion operation takes named sources. Each source is a ranked list from its own query, and fusion merges them.

{
  "ops": {
    "hybrid": {
      "fusion": {
        "sources": {
          "lexical": {
            "query": { "match": { "title_t": "dune" } },
            "limit": 100
          },
          "semantic": {
            "query": {
              "knn": { "field": "embedding_v", "query": [0.75, 0.15, 0.10], "k": 100 }
            },
            "limit": 100
          }
        },
        "filter": ["kind_s:fiction"],
        "rrf": { "k": 60 },
        "limit": 10,
        "get_scores": true,
        "fields": ["id", "title_t"]
      }
    }
  }
}

The result stays under the name you gave the operation, here ops.hybrid, with the fused documents and scores. A fusion has no shorthand form at the root of a request, so request and response both use named ops.

Choose exactly one of rrf or linear.

#Reciprocal rank fusion

With RRF, each document scores weight / (k + rank) for every source that returned it, summed over sources, with ranks starting at 1. It uses only ranks, so BM25 scores and cosine similarities never need to share a scale. That makes it a robust default that needs no tuning.

A document ranked first in both sources above scores 1/61 + 1/61. A document found by only one source still appears, with a smaller score.

#Weights

Weights scale a source's contribution without changing which candidates it brings:

{
  "fusion": {
    "sources": {
      "lexical":  { "query": { "match": { "title_t": "dune" } }, "limit": 100 },
      "semantic": { "query": { "knn": { "field": "embedding_v", "query": [0.75, 0.15, 0.10], "k": 100 } }, "limit": 100 }
    },
    "weights": { "lexical": 1, "semantic": 2 },
    "rrf": {}
  }
}

Weights must be finite and at least 0. Every key must name a source, and at least one weight must be above 0. A weight of 0 switches that source off completely: it is not run, and its documents do not reach the result. That lets you toggle a source from configuration without rewriting the request.

#Linear fusion

Linear fusion combines scores instead of ranks. Each source's scores are rescaled to the range 0 to 1 over its own result list, then summed with the weights:

normalized(d) = (score(d) - min) / (max - min)
fused(d)      = sum over sources of weight * normalized(d)

{
  "fusion": {
    "sources": {
      "lexical":  { "query": { "match": { "title_t": "dune" } }, "limit": 100 },
      "semantic": { "query": { "knn": { "field": "embedding_v", "query": [0.75, 0.15, 0.10], "k": 100 } }, "limit": 100 }
    },
    "weights": { "lexical": 0.3, "semantic": 0.7 },
    "linear": { "normalization": "min_max" }
  }
}

min and max are the lowest and highest scores in that source's list, so its best document gets 1 and its worst gets 0. A source that did not return a document contributes 0 to it. If all scores in a list are equal, every document in it gets 1.

Prefer RRF when score distributions are unknown or unstable, or when a source is sorted by a field. Prefer linear when score magnitudes carry information you want to keep, such as a clear winner in one source beating a near tie in another, and you can tune weights against relevance judgments. Min-max is sensitive to outliers and to the list size, so retune weights after changing a source's limit. Every source under linear fusion must rank by relevance score.

#Pools, filters and facets

• Pool size. A document outside a source's limit cannot contribute from that source. Size each limit for the recall your final limit needs. 50 to 200 is a common range.
• Paging. Fusion offset pages within the union of the source pools and does not enlarge them. Deep paging needs larger source limits. A per-source offset is ignored.
• Filters. The fusion-level filter applies to every source. A source can add its own on top. Filters apply inside the vector search too, not afterwards.
• Facets. Operations placed in the fusion's ops see the whole candidate set, which is every document in any source's list after filters. The fusion's limit and offset do not change facet counts. For facets over the full keyword match set, add a sibling top_docs operation at the request root instead.
• Diversity. A source can use collapse so its limit counts groups, and its best members enter fusion. fusion.collapse groups the fused list before paging. See Nearest groups and Searching.

#Elasticsearch and OpenSearch style

If your application already speaks Elasticsearch, add a knn section beside the query. A document matches if either part matches, and its score is the sum of the scores times their boosts:

curl -s https://api.warpsearch.io/compat/es/books/_search \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": { "match": { "title_t": { "query": "dune", "boost": 0.4 } } },
    "knn": {
      "field": "embedding_v",
      "query_vector": [0.75, 0.15, 0.10],
      "k": 50,
      "num_candidates": 200,
      "boost": 0.6,
      "filter": { "term": { "kind_s": "fiction" } }
    },
    "size": 10
  }'

A few differences from Elasticsearch to keep in mind. Vector scores are WarpSearch similarity scores (raw cosine or dot product), not Elasticsearch's rescaled values, so boosts that worked on one engine may need retuning. A knn section's own filter restricts only its candidates. To filter the keyword side too, put the filter inside the query as well.

The OpenSearch knn query clause works the same way inside a bool query, beside must and should clauses. On Solr, q={!knn f=embedding_v topK=50}[0.75, 0.15, 0.10] runs a vector search, and fq filters apply before the neighbors are chosen. See Elasticsearch compatibility, OpenSearch compatibility and Solr compatibility.

Summed scores are not rank fusion. If you want RRF or normalized linear combination, use the native fusion operation.

#Recipes

Start with the defaults, collect a handful of queries with known good answers, and change one thing at a time.

#Measure recall

Approximate vector search can miss true neighbors. To see whether that hurts your hybrid results, run the same request twice and set "exact": true on the knn in one of them. If the fused lists barely differ, raise your source limit or ivf.nprobe only as much as needed. If they differ a lot, see the tuning recipes in Vector search.

#Reranking

Not supported: cross-encoder and other reranker models, including Elasticsearch's text_similarity_reranker retriever. WarpSearch does not host models.

Do reranking in your application. Retrieve the top 50 to 100 fused results, send them to your reranker, and keep the best few. Because fusion already blends keywords and meaning, a reranker usually only has to reorder a short list.

#Not supported

Not supported: the Elasticsearch retriever syntax (including rrf and linear retrievers), and OpenSearch hybrid, neural and neural_sparse queries with search pipelines. They are rejected with an unsupported_feature error. Use the native fusion operation, or a knn section beside a query.

Coming soon: sparse vectors for learned sparse retrieval (for example SPLADE) and rank_features fields, which suit learned token weights and static signals such as popularity. They are in development, with no release date. Once available, a sparse source can join the same fusion as the keyword and dense sources.

For the vector side of the story, see Vector search and Multi-vector fields.

View this page as Markdown