For AI agents: this page is also available as Markdown at /docs/highlighting.md. Every docs page is: add .md to its URL. llms.txt lists them all.

Highlighting

Highlighting returns excerpts of a document with matching words marked. Add highlight to a standard search request. You can retrieve only an ID and title while highlighting a longer body; the highlighted field does not have to be in the returned fields list.

#Native API

Send a normal request to POST /v1/collections/articles/search:

{
  "language": "lucene",
  "query": "body_t:climate",
  "fields": ["id", "title"],
  "highlight": {
    "fields": ["body_t"],
    "backend": "auto",
    "fragment_size": 150,
    "fragments": 2
  }
}

The native response puts snippets in each hit's highlight map, alongside fields:

{
  "id": "123",
  "fields": {"title": "Weather report"},
  "highlight": {
    "body_t": ["Changing <em>climate</em> affects rainfall."]
  }
}

Fields without a match are omitted unless no_match_size requests a fallback prefix. The highlight option also works with the other /v1 query languages and the MCP search tool. For a native WarpSearch expression, use the same options with language: "warpsearch":

{
  "language": "warpsearch",
  "query": "body_t:$term",
  "vars": {"term": "climate"},
  "fields": ["id"],
  "highlight": {"fields": ["body_t"], "backend": "analysis"}
}

Native expression access rules still apply. Highlighted searches currently have no signed /translate replay form; send the original request again. Field aliases and field permissions apply to highlighting, including match_fields and highlight_query. Wildcard field selectors are unavailable on collections with restricted field lists.

If calling the underlying engine's /collections/{collection}/_search directly, omit the language wrapper and put the query and highlight in the same request. Its result column is named _highlight_; the gateway converts that column to the highlight map shown above.

#Choose a backend

Explicit selections do not silently fall back. An unavailable backend returns an error. Positions are unavailable on docs-only keyword fields and fields with index: "none". No backend depends on norms; there is currently no separate text-field norms toggle in the native schema. auto keeps the positions optimization opt-in. Stored chunks must still be read and decompressed, and character filters may process the full source; benchmark your workload before choosing a backend for latency reasons.

Ordinary text fields store source by default. A string or id field needs stored source to be highlighted; matching a keyword highlights the whole value. A field with stored: false cannot be highlighted from its postings alone.

#Configure stored offsets

Create the field through the native schema API before ingesting documents. For example, PUT /v1/collections/articles with:

{
  "schema": {
    "fields": {
      "body": {
        "type": "highlight_text",
        "params": {"analyzer": "english", "stored_offsets": true}
      }
    }
  }
}

Index body as a string or an array of strings. Named analyzer presets are supported, including standard (the default), english, and enabled analysis plugin presets. Inline custom analyzer chains are not supported by this field type.

Through the gateway, query and highlight the logical name body:

{
  "language": "lucene",
  "query": "body:climate",
  "fields": ["id", "body"],
  "highlight": {"fields": ["body"], "backend": "stored_offsets"}
}

The gateway resolves lexical queries to the field's private text representation. Clients do not name its internal $text or $data fields. Raw warpsearch expressions are opaque to the gateway and do not get this field rewrite; use a translated language such as lucene or es for queries on highlight_text fields.

stored_offsets: false keeps source and indexed positions, so analysis and positions remain available. Enabling stored offsets or changing the analyzer requires reindexing. Existing plain text fields do not gain offsets retroactively. Elasticsearch mapping options such as term_vector and index_options do not configure this plugin storage; use the native field configuration above.

#Fragment and rendering options

Use defaults at the top level and override them inside a field's object:

{
  "language": "lucene",
  "query": "title:climate OR body_t:climate",
  "highlight": {
    "encoder": "html",
    "fields": {
      "title": {"backend": "analysis", "fragments": 0},
      "body_t": {"backend": "positions", "fragment_size": 150, "fragments": 2}
    }
  }
}

All queried variants of a logical field contribute source-offset matches. Fragments never join separate array values. Phrase occurrences are verified, while negative clauses and separately supplied filters do not contribute spans. Highlighting extracts positive lexical matches; it is not a complete explanation of Boolean matching or vector similarity. For vector-only searches, supply a lexical highlight_query if you want highlighted text.

#Source offsets

format: "offsets" returns unescaped source text and spans instead of injecting markup:

{
  "text": "a cat",
  "value_index": 0,
  "start": 10,
  "end": 15,
  "spans": [[2, 5]]
}

start and end are half-open UTF-8 byte offsets into the original value. Each span is a half-open UTF-8 byte range relative to text. value_index identifies the original array value. Fragment sizes and analysis limits count code points, not bytes or JavaScript UTF-16 units.

#Elasticsearch and OpenSearch

Use the normal highlight object on POST /compat/es/articles/_search or POST /compat/opensearch/articles/_search:

{
  "query": {"match": {"body_t": "climate"}},
  "_source": ["title"],
  "highlight": {
    "type": "unified",
    "encoder": "html",
    "pre_tags": ["<mark>"],
    "post_tags": ["</mark>"],
    "fields": {
      "body_t": {"fragment_size": 150, "number_of_fragments": 2}
    }
  }
}

Each entry in hits.hits carries a highlight map. It is independent of _source and is returned even with _source: false. _msearch, hybrid/fusion searches, collapsed inner hits, and top_hits aggregations support highlighting too. Grafana's Elasticsearch/OpenSearch requests use the same compatibility endpoints.

These type names select WarpSearch backends, not Lucene implementations. Fragment boundaries, scoring, and character-limit semantics can differ. Semantic highlighters, multiple tag pairs, tags_schema, phrase_limit, fragmenter, fragment_offset, custom boundary characters, locale-specific boundary scanners, and per-field highlight_query are rejected with unsupported_feature, rather than silently ignored. Highlighting does not add support for otherwise unsupported query types such as nested-document queries.

#Solr and SolrJ

Use standard Solr highlighting parameters on /compat/solr/articles/select:

curl 'https://api.warpsearch.io/compat/solr/articles/select?q=body_t:climate&fl=id,title&hl=true&hl.fl=body_t&hl.snippets=2&hl.fragsize=150&hl.encoder=html' \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY"

Solr responses contain the usual separate highlighting object, keyed by document ID:

{
  "highlighting": {
    "123": {"body_t": ["Changing <em>climate</em> affects rainfall."]}
  }
}

The JSON Request API accepts the same settings inside params. Form-encoded requests and wt=javabin are supported; SolrJ receives the corresponding named-list structure.

Per-field forms such as f.body_t.hl.fragsize=200 are supported. Grouped document lists can carry highlights. hl.usePhraseHighlighter and hl.highlightMultiTerm must remain true. Alternate fields, custom fragmenters/formatters, scoring tunables, and other unmapped hl.* settings are rejected. Solr method names select the backends above; they do not reproduce Solr's fragment scoring exactly.

#Other query languages and limits

SQL, PostgreSQL-style SQL, Lucene, Grafana models and LogQL searches can request native highlight through /v1. PostgreSQL's ts_headline() SELECT expression remains unsupported; use the request option instead. There is no Loki HTTP API, and the turbopuffer compatibility API does not define a highlighting option.

The plugin requires matching gateway protobufs and a node build containing the highlighter. It runs after paging and does not change document membership, order or scores. Normal plan limits, collection permissions and timeouts still apply. Highlight queries count toward query byte, clause and depth limits. At most 64 highlighted field selectors and 100 fragments per field are allowed; combined tags are limited to 4096 bytes. Engine-side expansion can also reject a wildcard selecting more than 64 eligible fields.

Work is bounded by source, token, phrase and passage-selection limits. Request-level native max_expansions (default 4096) and max_terms_examined (default 100000) bound dictionary work. Exceeding these limits fails explicitly. Source truncation returns a warning and never invents a match from a partial token. See Limits and errors.

View this page as Markdown