For AI agents: this page is also available as Markdown at /docs/suffix-search.md. Every docs page is: add .md to its URL. llms.txt lists them all.

Suffix search

A wildcard query with a literal start, such as install*, jumps to that part of the term dictionary. One that starts with *, such as *.exe or *\install_update.bat, has nowhere to start and reads every term of the field. On a field with millions of distinct values, such as file paths or command lines, that is slow and can exceed the query's term budget.

The fix, as in Solr's ReversedWildcardFilter, is to index a reversed copy of every term, so a suffix becomes a prefix. The reverse filter writes the reversed copy into a variant of the field, and the rwildcard query runs each pattern on whichever side has the longer literal anchor.

#Set up the field

Give the field a rev variant of the same type whose normalizer or analyzer ends with reverse:

{
  "fields": {
    "cmd": {
      "type": "string",
      "variants": {"rev": {"type": "string", "normalizer": ["reverse"]}}
    }
  }
}

The variant indexes each value reversed as cmd__rev. Keep the variant's type equal to the field's, and put reverse last, after any lowercasing or folding, so both sides are normalized the same way. For case-insensitive matching, use ["lowercase"] on the field and ["lowercase", "reverse"] on the variant.

Adding a variant to an existing field does not reverse older documents; reindex them.

#Query

rwildcard takes the forward field and a pattern, in Elasticsearch Query DSL. On the WarpSearch API set "language": "es"; on the Elasticsearch and OpenSearch endpoints use it like any other query.

With these documents:

[
  {"id": "r1", "cmd": "C:\\Windows\\install_update.bat"},
  {"id": "r2", "cmd": "C:\\Windows\\System32\\cmd.exe /c install_update.bat"},
  {"id": "r3", "cmd": "C:\\Apps\\webclient.exe"},
  {"id": "r4", "cmd": "WebClient.EXE"},
  {"id": "r5", "cmd": "D:\\tools\\webclient"},
  {"id": "r6", "cmd": "notepad.exe notes.txt"}
]

curl -X POST https://api.warpsearch.io/v1/collections/processes/search \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "language": "es",
    "query": {"rwildcard": {"field": "cmd", "pattern": "*install_update.bat"}},
    "fields": ["id"]
  }'

This returns r1 and r2. It runs as the prefix tab.etadpu_llatsni* on cmd__rev.

Combine rwildcard with other clauses in a bool query, or put it in filters, as with any query. It takes boost and _name too. rwildcard has no Lucene or SQL form.

rwildcard counts as a wildcard query for your plan: plans without wildcard queries get 403 forbidden. See Limits and errors.

#Which side runs

The query counts the literal characters before the first * or ?, in the pattern as written and reversed, and runs on the side with more, as a wildcard on that field. Ties run forward, so the reversed variant is used only when it helps. A pattern with no literal at either end, such as *foo*, gains nothing and runs forward: for those, see Substring search.

#Words or whole values

On a string field the whole value is reversed, so a suffix matches the end of the value. On a text field each word is reversed, so a suffix matches the end of a word, even when more words follow. Choose the type by which end you mean:

{
  "fields": {
    "args": {
      "type": "text",
      "analyzer": {"tokenizer": "whitespace", "filters": ["lowercase"]},
      "variants": {
        "rev": {
          "type": "text",
          "analyzer": {"tokenizer": "whitespace", "filters": ["lowercase", {"name": "reverse", "params": {"marker": "#"}}]}
        }
      }
    }
  }
}

For {"id": "r7", "args": "up.bat --quiet"}, the query {"rwildcard": {"field": "args", "suffix": ".bat", "marker": "#"}} matches, because the word up.bat ends in .bat.

The optional marker (one character) is put before every reversed word, so reversed words can never match forward text in the same index. If the variant sets one, pass the same marker to the query. Markers work only in text analyzers, not in string normalizers.

#Costs and limits

• The variant doubles the field's term dictionary and postings.
• Each side is an indexed term, so a value longer than 255 bytes is cut on both: the forward term keeps the start and the reversed term keeps the end. Matching the middle of long values needs Substring search.
• Only wildcard patterns are rewritten. Regex and fuzzy queries run as usual.
• A broad suffix such as *e still matches many terms and can still exceed the term budget.
• reverse works on whole characters, so any script is reversed correctly.

View this page as Markdown