# Tuning relevance

Relevance tuning is changing how results are scored until the best documents come first. This page
covers the controls WarpSearch gives you, from the field's own scoring settings to hand-written
boosts, and then how to tell whether a change helped. For the ideas behind scoring, read
[Full-text search](full-text-search.md) first.

Tune in this order, and stop when the results are good enough:

1. Fix [analysis](text-analysis.md), so the right documents match at all.
2. Weight the fields, so matches in important fields count for more.
3. Tune BM25 per field.
4. Add manual boosts and function scores for business rules.
5. Measure with a judgment list, so you know a change helped across many queries.

## BM25 per field: k1 and b

Two field options shape how BM25 scores a `text` field. Set them in the schema like any other
option:

```json
{"fields": {"body": {"type": "text", "k1": 1.5, "b": 0.4}}}
```

| Option | Default | Range | Applies to | Meaning |
|---|---|---|---|---|
| `k1` | 1.2 | 0 or more | `text`, `string`, `id` | Term frequency saturation. Higher values let repeated terms keep adding to the score. `0` scores presence only, so how often the term appears does not matter. |
| `b` | 0.75 | 0 to 1 | `text` | Length normalization. `0` ignores the field's length, `1` normalizes fully, so a match in a short field counts for more than the same match in a long one. |

Setting `b` on a field type other than `text` is a schema error, because other types have no
length to normalize.

Both options are applied when a query runs, so you can change them on a collection that already has
data and search again straight away, with no reindex. Like other field options they are inherited
from a `parent` or template, and each [variant](schemas-and-fields.md#variants) can set its own.
They apply to every scored text query on the field, including `match`, `term`, phrase,
`multi_match` and fuzzy queries. They are set per field, not per query.

Some rules of thumb:

- **Titles and names:** lower `b`, such as 0.2, or `0`. Their length rarely says anything about
  relevance, and one extra word should not sink a document.
- **Long bodies:** keep `b` near the default. A passing mention in a very long page is weaker
  evidence than the same mention in a short one.
- **Repetition is a signal:** raise `k1`, such as 2, when a document that uses a word many times is
  really more about it, as in long reference text.
- **Repetition is noise:** lower `k1`, or use `0`, for short fields such as tags or titles, where
  saying a word twice adds nothing.

For example, searching `dune` over a field holding the titles "Dune" and "The Dune Encyclopedia:
A Complete Guide to the Universe of Dune" with the default `b` ranks the short title first, since
its length is far below average. With `"b": 0` the two titles differ only by how often `dune`
occurs, and `k1` decides how much the second occurrence adds. Change one option, search again, and
compare.

## Weighting fields in multi_match

A `multi_match` query searches several fields at once. Give each field a `weight` to say how much
its matches count:

```json
{"multi_match": {
  "text": "dune messiah",
  "fields": [{"field": "title_t", "weight": 2}, {"field": "description_t"}],
  "mode": "combined",
  "operator": "and",
  "tie": 0.3
}}
```

A weight is a number above 0, and a field without one has weight 1. The shorthand
`"title_t^2"` also works. If a field is listed twice, the larger weight is kept. A field name can
use a wildcard, up to 64 matching fields.

`mode` chooses how the fields combine:

- **`combined`** (the default) scores text fields as if they were one field (BM25F). A text
  weight scales the field's term frequency and length, so a weight of 2 on a title behaves as if
  every word in it counted twice. Fields are only scored together when they share the same
  analyzer, `long_terms` setting, `k1` and `b`. Fields that differ in any of these are scored as
  separate groups, and the groups are then joined with `tie`. A weight on a field of another type
  multiplies its score.
- **`per_field`** scores each field on its own and multiplies the score by its weight.

On the [Elasticsearch-compatible API](compat-elasticsearch.md#query-dsl) the same controls are
`multi_match` `type` (`best_fields` and `phrase` are `per_field`, `cross_fields` and
`combined_fields` are `combined`), `field^boost` and `tie_breaker`.

`tie` controls how the field or group scores join: the result is the best score plus
`tie` times the rest, that is `best + tie * (sum - best)`. The default `1` adds the scores up, so a
document matching in many fields wins. `0` keeps only the best field. A value like `0.3` is a
common middle: the best field leads, and other matching fields break ties.

## Manual boosting

Sometimes you know something the text cannot tell: a sponsored product, a newer edition, an
in-stock item. Manual boosts change the score directly.

**Boost a query.** Wrap any query in `boost` to multiply every score it produces. The default is
`1`, and values from 0 up are accepted, so `0.5` demotes and `2` promotes:

```json
{"boost": {"query": {"match": {"title_t": "dune"}}, "boost": 2.0}}
```

Queries that take a `boost` of their own, such as `match`, accept it inline, as in
`{"match": {"title_t": {"query": "dune", "boost": 2}}}`. In a `bool` query this is how you rank
some clauses above others: put the strong evidence in a boosted `should` clause.

**Take the best of several queries.** `dis_max` scores a document by its best matching query,
which stops a document from winning only because it matched many weak clauses. Its `tie` works as
in `multi_match`, and defaults to `0`, which uses only the best score. `1` adds them all up:

```json
{"dis_max": {
  "queries": [{"match": {"title_t": "dune"}}, {"match": {"description_t": "dune"}}],
  "tie": 0.3
}}
```

**Keep filters out of the score.** A condition that is only a rule, such as `in_stock`, belongs in
a filter, which narrows results without changing any score. See
[Filters versus query](searching.md#filters-versus-query).

## Function scores

A `function_score` query changes a query's score with values from the document, so business signals
such as popularity or freshness count alongside the text match. It is available in the
Elasticsearch language and on the [Elasticsearch-compatible API](compat-elasticsearch.md). It
supports `weight`, `field_value_factor`, decay functions, `random_score` and functions that apply
only to documents matching a filter.

This query ranks matches for `dune` by text relevance, scaled by the log of a `popularity`
field, and favors documents published recently:

```json
{"function_score": {
  "query": {"match": {"title_t": "dune"}},
  "functions": [
    {"field_value_factor": {"field": "popularity", "modifier": "log1p", "missing": 0}},
    {"gauss": {"published": {"origin": "now", "scale": "365d", "decay": 0.5}}}
  ],
  "score_mode": "multiply",
  "boost_mode": "multiply"
}}
```

How to choose:

| You want | Use |
|---|---|
| A fixed lift for documents matching a condition | A function with a `filter` and a `weight`. |
| Popular or highly rated items to rank higher | `field_value_factor` with a damping `modifier` such as `log1p`, so a huge value does not swamp the text score. |
| Newer or closer documents to rank higher, fading gradually | A decay function (`gauss`, `exp` or `linear`) on a date or geo field. |
| Variety between sessions | `random_score` with a `seed`. |

Keep boosts small relative to the text score, and damp numeric signals. A boost large enough to
override relevance turns search into a sorted list, and it is easy to hide a real problem with
analysis behind one.

## Offline evaluation with judgment lists

Changing a boost for one query usually changes results for many others. To tune with confidence,
measure, using a **judgment list**: a fixed set of queries, each paired with the documents that
should be found, and how good each one is.

**Build the list.**

1. Collect 50 to 200 real queries from your search logs. Include the frequent ones, the ones with
   no clicks, and a sample of long or unusual ones.
2. For each query, record the documents that a good result list would contain and grade each, for
   example 0 (irrelevant), 1 (related), 2 (good) and 3 (perfect). Take grades from people who
   know your content, and from click and purchase data where you have it.
3. Keep the list in version control as plain data: query, document id, grade.

**Measure a configuration.** Run every query against your collection, with the settings under
test, and score the returned ids against the grades:

| Metric | Answers | Use it when |
|---|---|---|
| Precision at k | What share of the top k results are relevant? | Users scan only the first screen. |
| Recall at k | What share of all relevant documents appear in the top k? | You feed results to a reranker or a language model, as in RAG. |
| MRR (mean reciprocal rank) | How high is the first relevant result? | Users want one right answer, as in navigational search. |
| nDCG at k | Are the best documents ranked first, given graded judgments? | You have graded judgments and care about the whole order. |

A short script is enough: send each query to the [search API](searching.md), read the returned
ids, and compute the metric. Run the same script before and after each change.

**Work the loop.**

- Change one setting at a time, such as a field weight, `k1` or `b`, a boost or the
  [hybrid](hybrid-search.md) weights, and re-run the whole list. Keep a change only if the metric
  rises overall, and look at the queries that got worse.
- Hold some queries back. If you tune on every query, the settings fit that list but may not
  generalize. Tune on most of them, and check the held-out ones at the end.
- Refresh the list as your catalog and your users' queries change, and add each failure you find
  in production as a new row.
- Compare results from the same data. Scores are only meaningful within one query, so compare
  ranks and metrics, not raw score values.

A judgment list also settles arguments. When someone reports that "this search is bad", add the
query with the right answers, and the next tuning round has to get it right without breaking the
others.

## Where to go next

1. [Full-text search](full-text-search.md) for the concepts behind scoring.
2. [Schemas and field types](schemas-and-fields.md#field-options) for every field option.
3. [Hybrid search](hybrid-search.md) to balance keyword and vector scores.
4. [Searching](searching.md) for filters, sorting and paging.