For AI agents: this page is also available as Markdown at /docs/tuning-relevance.md. Every docs page is: add .md to its URL. llms.txt lists them all.

Tuning relevance

Relevance tuning is changing how results are scored until the best documents come first. This page covers the controls WarpSearch gives you, from the field's own scoring settings to hand-written boosts, and then how to tell whether a change helped. For the ideas behind scoring, read Full-text search first.

Tune in this order, and stop when the results are good enough:

1. Fix analysis, so the right documents match at all.
2. Weight the fields, so matches in important fields count for more.
3. Tune BM25 per field.
4. Add manual boosts and function scores for business rules.
5. Measure with a judgment list, so you know a change helped across many queries.

#BM25 per field: k1 and b

Two field options shape how BM25 scores a text field. Set them in the schema like any other option:

{"fields": {"body": {"type": "text", "k1": 1.5, "b": 0.4}}}

Setting b on a field type other than text is a schema error, because other types have no length to normalize.

Both options are applied when a query runs, so you can change them on a collection that already has data and search again straight away, with no reindex. Like other field options they are inherited from a parent or template, and each variant can set its own. They apply to every scored text query on the field, including match, term, phrase, multi_match and fuzzy queries. They are set per field, not per query.

Some rules of thumb:

• Titles and names: lower b, such as 0.2, or 0. Their length rarely says anything about relevance, and one extra word should not sink a document.
• Long bodies: keep b near the default. A passing mention in a very long page is weaker evidence than the same mention in a short one.
• Repetition is a signal: raise k1, such as 2, when a document that uses a word many times is really more about it, as in long reference text.
• Repetition is noise: lower k1, or use 0, for short fields such as tags or titles, where saying a word twice adds nothing.

For example, searching dune over a field holding the titles "Dune" and "The Dune Encyclopedia: A Complete Guide to the Universe of Dune" with the default b ranks the short title first, since its length is far below average. With "b": 0 the two titles differ only by how often dune occurs, and k1 decides how much the second occurrence adds. Change one option, search again, and compare.

#Weighting fields in multi_match

A multi_match query searches several fields at once. Give each field a weight to say how much its matches count:

{"multi_match": {
  "text": "dune messiah",
  "fields": [{"field": "title_t", "weight": 2}, {"field": "description_t"}],
  "mode": "combined",
  "operator": "and",
  "tie": 0.3
}}

A weight is a number above 0, and a field without one has weight 1. The shorthand "title_t^2" also works. If a field is listed twice, the larger weight is kept. A field name can use a wildcard, up to 64 matching fields.

mode chooses how the fields combine:

• combined (the default) scores text fields as if they were one field (BM25F). A text weight scales the field's term frequency and length, so a weight of 2 on a title behaves as if every word in it counted twice. Fields are only scored together when they share the same analyzer, long_terms setting, k1 and b. Fields that differ in any of these are scored as separate groups, and the groups are then joined with tie. A weight on a field of another type multiplies its score.
• per_field scores each field on its own and multiplies the score by its weight.

On the Elasticsearch-compatible API the same controls are multi_match type (best_fields and phrase are per_field, cross_fields and combined_fields are combined), field^boost and tie_breaker.

tie controls how the field or group scores join: the result is the best score plus tie times the rest, that is best + tie * (sum - best). The default 1 adds the scores up, so a document matching in many fields wins. 0 keeps only the best field. A value like 0.3 is a common middle: the best field leads, and other matching fields break ties.

#Manual boosting

Sometimes you know something the text cannot tell: a sponsored product, a newer edition, an in-stock item. Manual boosts change the score directly.

Boost a query. Wrap any query in boost to multiply every score it produces. The default is 1, and values from 0 up are accepted, so 0.5 demotes and 2 promotes:

{"boost": {"query": {"match": {"title_t": "dune"}}, "boost": 2.0}}

Queries that take a boost of their own, such as match, accept it inline, as in {"match": {"title_t": {"query": "dune", "boost": 2}}}. In a bool query this is how you rank some clauses above others: put the strong evidence in a boosted should clause.

Take the best of several queries. dis_max scores a document by its best matching query, which stops a document from winning only because it matched many weak clauses. Its tie works as in multi_match, and defaults to 0, which uses only the best score. 1 adds them all up:

{"dis_max": {
  "queries": [{"match": {"title_t": "dune"}}, {"match": {"description_t": "dune"}}],
  "tie": 0.3
}}

Keep filters out of the score. A condition that is only a rule, such as in_stock, belongs in a filter, which narrows results without changing any score. See Filters versus query.

#Function scores

A function_score query changes a query's score with values from the document, so business signals such as popularity or freshness count alongside the text match. It is available in the Elasticsearch language and on the Elasticsearch-compatible API. It supports weight, field_value_factor, decay functions, random_score and functions that apply only to documents matching a filter.

This query ranks matches for dune by text relevance, scaled by the log of a popularity field, and favors documents published recently:

{"function_score": {
  "query": {"match": {"title_t": "dune"}},
  "functions": [
    {"field_value_factor": {"field": "popularity", "modifier": "log1p", "missing": 0}},
    {"gauss": {"published": {"origin": "now", "scale": "365d", "decay": 0.5}}}
  ],
  "score_mode": "multiply",
  "boost_mode": "multiply"
}}

How to choose:

Keep boosts small relative to the text score, and damp numeric signals. A boost large enough to override relevance turns search into a sorted list, and it is easy to hide a real problem with analysis behind one.

#Offline evaluation with judgment lists

Changing a boost for one query usually changes results for many others. To tune with confidence, measure, using a judgment list: a fixed set of queries, each paired with the documents that should be found, and how good each one is.

Build the list.

1. Collect 50 to 200 real queries from your search logs. Include the frequent ones, the ones with no clicks, and a sample of long or unusual ones.
2. For each query, record the documents that a good result list would contain and grade each, for example 0 (irrelevant), 1 (related), 2 (good) and 3 (perfect). Take grades from people who know your content, and from click and purchase data where you have it.
3. Keep the list in version control as plain data: query, document id, grade.

Measure a configuration. Run every query against your collection, with the settings under test, and score the returned ids against the grades:

A short script is enough: send each query to the search API, read the returned ids, and compute the metric. Run the same script before and after each change.

Work the loop.

• Change one setting at a time, such as a field weight, k1 or b, a boost or the hybrid weights, and re-run the whole list. Keep a change only if the metric rises overall, and look at the queries that got worse.
• Hold some queries back. If you tune on every query, the settings fit that list but may not generalize. Tune on most of them, and check the held-out ones at the end.
• Refresh the list as your catalog and your users' queries change, and add each failure you find in production as a new row.
• Compare results from the same data. Scores are only meaningful within one query, so compare ranks and metrics, not raw score values.

A judgment list also settles arguments. When someone reports that "this search is bad", add the query with the right answers, and the next tuning round has to get it right without breaking the others.

#Where to go next

1. Full-text search for the concepts behind scoring.
2. Schemas and field types for every field option.
3. Hybrid search to balance keyword and vector scores.
4. Searching for filters, sorting and paging.

View this page as Markdown