# Suffix search

A wildcard query with a literal start, such as `install*`, jumps to that part of the term
dictionary. One that starts with `*`, such as `*.exe` or `*\install_update.bat`, has nowhere to
start and reads every term of the field. On a field with millions of distinct values, such as file
paths or command lines, that is slow and can exceed the query's term budget.

The fix, as in Solr's `ReversedWildcardFilter`, is to index a reversed copy of every term, so a suffix
becomes a prefix. The `reverse` filter writes the reversed copy into a
[variant](schemas-and-fields.md#variants) of the field, and the `rwildcard` query runs each
pattern on whichever side has the longer literal anchor.

## Set up the field

Give the field a `rev` variant of the same type whose normalizer or analyzer ends with `reverse`:

```json
{
  "fields": {
    "cmd": {
      "type": "string",
      "variants": {"rev": {"type": "string", "normalizer": ["reverse"]}}
    }
  }
}
```

The variant indexes each value reversed as `cmd__rev`. Keep the variant's type equal to the
field's, and put `reverse` last, after any lowercasing or folding, so both sides are normalized
the same way. For case-insensitive matching, use `["lowercase"]` on the field and
`["lowercase", "reverse"]` on the variant.

Adding a variant to an existing field does not reverse older documents; reindex them.

## Query

`rwildcard` takes the forward field and a pattern, in Elasticsearch Query DSL. On the WarpSearch
API set `"language": "es"`; on the [Elasticsearch](compat-elasticsearch.md) and
[OpenSearch](compat-opensearch.md) endpoints use it like any other query.

With these documents:

```json
[
  {"id": "r1", "cmd": "C:\\Windows\\install_update.bat"},
  {"id": "r2", "cmd": "C:\\Windows\\System32\\cmd.exe /c install_update.bat"},
  {"id": "r3", "cmd": "C:\\Apps\\webclient.exe"},
  {"id": "r4", "cmd": "WebClient.EXE"},
  {"id": "r5", "cmd": "D:\\tools\\webclient"},
  {"id": "r6", "cmd": "notepad.exe notes.txt"}
]
```

```bash
curl -X POST https://api.warpsearch.io/v1/collections/processes/search \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "language": "es",
    "query": {"rwildcard": {"field": "cmd", "pattern": "*install_update.bat"}},
    "fields": ["id"]
  }'
```

This returns r1 and r2. It runs as the prefix `tab.etadpu_llatsni*` on `cmd__rev`.

| Query | Matches | Why |
|---|---|---|
| `{"rwildcard": {"field": "cmd", "suffix": "install_update.bat"}}` | r1, r2 | The same as the pattern above, written as a literal suffix. |
| `{"rwildcard": {"field": "cmd", "pattern": "*.exe"}}` | r3 | The whole value ends in lowercase `.exe`. r6 does not: arguments follow the executable. |
| `{"rwildcard": {"field": "cmd", "pattern": "*.exe", "case_insensitive": true}}` | r3, r4 | Also accepts `.EXE`. |
| `{"rwildcard": {"field": "cmd", "pattern": "*web*client"}}` | r5 | Ends in `client`, with `web` earlier. |
| `{"rwildcard": {"field": "cmd", "pattern": "notepad*"}}` | r6 | The forward side has the longer anchor, so it runs on `cmd`. |

Combine `rwildcard` with other clauses in a `bool` query, or put it in `filters`, as with any
query. It takes `boost` and `_name` too. `rwildcard` has no Lucene or SQL form.

| Param | Meaning |
|---|---|
| `field` | The forward `string` or `text` field. Required. |
| `pattern` | A wildcard pattern: `*` matches any characters, `?` one character, `\` escapes. |
| `suffix` | A literal suffix, the same as `pattern` `*` plus the escaped suffix. Give exactly one of `pattern` and `suffix`. |
| `case_insensitive` | Default `false`. Match letters in any case. |
| `reversed` | The variant that holds the reversed terms, of the same type as `field`. Default `<field>__rev`. |
| `marker` | The `reverse` filter's marker, if the variant sets one. |

`rwildcard` counts as a wildcard query for your plan: plans without wildcard queries get
`403 forbidden`. See [Limits and errors](limits-and-errors.md).

## Which side runs

The query counts the literal characters before the first `*` or `?`, in the pattern as written and
reversed, and runs on the side with more, as a wildcard on that field. Ties run forward, so the
reversed variant is used only when it helps. A pattern with no literal at either end, such as
`*foo*`, gains nothing and runs forward: for those, see [Substring search](substring-search.md).

## Words or whole values

On a `string` field the whole value is reversed, so a suffix matches the end of the value. On a
`text` field each word is reversed, so a suffix matches the end of a word, even when more words
follow. Choose the type by which end you mean:

```json
{
  "fields": {
    "args": {
      "type": "text",
      "analyzer": {"tokenizer": "whitespace", "filters": ["lowercase"]},
      "variants": {
        "rev": {
          "type": "text",
          "analyzer": {"tokenizer": "whitespace", "filters": ["lowercase", {"name": "reverse", "params": {"marker": "#"}}]}
        }
      }
    }
  }
}
```

For `{"id": "r7", "args": "up.bat --quiet"}`, the query
`{"rwildcard": {"field": "args", "suffix": ".bat", "marker": "#"}}` matches, because the word
`up.bat` ends in `.bat`.

The optional `marker` (one character) is put before every reversed word, so reversed words can
never match forward text in the same index. If the variant sets one, pass the same `marker` to the
query. Markers work only in `text` analyzers, not in `string` normalizers.

## Costs and limits

- The variant doubles the field's term dictionary and postings.
- Each side is an indexed term, so a value longer than 255 bytes is cut on both: the forward term
  keeps the start and the reversed term keeps the end. Matching the middle of long values needs
  [Substring search](substring-search.md).
- Only wildcard patterns are rewritten. Regex and fuzzy queries run as usual.
- A broad suffix such as `*e` still matches many terms and can still exceed the term budget.
- `reverse` works on whole characters, so any script is reversed correctly.