# Hebrew

Hebrew attaches prepositions, conjunctions and the definite article to the word that follows
(`לבית`, "to the house"), inflects nouns and verbs heavily, and is usually written without vowel
points, with extra `ו` and `י` letters standing in for them. A plain word index treats each of
these forms as a different word. The `hebrew` analyzer looks every word up in a Hebrew dictionary
of 337,000 words and indexes its lemmas, so a search for `בית` finds `לבית`, `ובבית` and `הבית`,
and a search for `ספר` finds `הספרים`.

```json
{"fields": {"body_he": {"type": "text", "analyzer": "hebrew"}}}
```

It is a port of HebMorph, the analyzer of the Elasticsearch Hebrew plugin, with its dictionary
built in. No license key or separate dictionary download is needed.

## What gets indexed

For each word the analyzer removes niqqud (vowel points), then indexes two kinds of terms at the
same position:

- **The exact form**, with a `$` suffix: `הספרים$`. It matches only that spelling.
- **Every lemma** the dictionary gives for the word, with its prefixes removed: `ספר`. A word
  with several readings gets several lemmas: `שמלות` is "dresses" (`שמלה`) or "that from"
  (`ש` + `מלות`, lemma `מל`).

For the sentence `הילדים הלכו לבית הספר` ("the children went to school"):

| Word | Exact form | Lemmas |
|---|---|---|
| `הילדים` | `הילדים$` | `ילד` |
| `הלכו` | `הלכו$` | `הלך`, `הילך` |
| `לבית` | `לבית$` | `בית` |
| `הספר` | `הספר$` | `ספר` |

So `ילד`, `ילדים` and `הילד` all find this document, and so do `בית` and `ספר`.

Other words:

- **Spelling without niqqud** is tolerated: `אמא` is looked up as `אימא` too, and an extra or
  missing `ו` or `י` still finds the lemma. Words over 20 letters are not tolerated.
- **Words not in the dictionary**, such as most names, are their own lemma: `book` gives `book$`
  and `book`.
- **Acronyms** written with gershayim (`צה"ל`) and words with a geresh (`ג'ירפה`) stay one word.
- **Words starting with a digit** are indexed as they are, with no exact form and no lemmas.
- Case and Unicode width are folded first, so Latin words match in any case.

## Exact matches

A query with the same analyzer produces the exact form and the lemmas of each query word, so it
matches every inflection. Documents that use the query's exact spelling also match its `$` term,
which usually ranks them higher. To match only the exact spelling, search the `$` form with a term query, which
is not analyzed:

```json
{"language": "es", "query": {"term": {"body_he": "הספרים$"}}}
```

Write the term without niqqud, as it is indexed.

## Prefix, wildcard and fuzzy terms

`hebrew_lemmatizer` does not run on prefix, wildcard and fuzzy terms (lemmatizing part of a word
means nothing), so they match the indexed terms as written: `ספר*` matches the lemma `ספר` and
exact forms that start with `ספר`, but not `הספרים$`. The terms are still case folded.

## The filter and the presets

| Name | Kind | Does |
|---|---|---|
| `hebrew` | analyzer | `unicode_word`, `nfkc_cf`, `fold`, `hebrew_lemmatizer`. Use it at index and query time. |
| `hebrew_query` | analyzer | The same chain. The name is kept so Elasticsearch mappings that use it carry over. |
| `hebrew_lemmatizer` | filter | Removes niqqud and emits the exact form and the lemmas. Param `suffix` (default `$`) marks the exact form; `""` emits it without a marker. |

To combine Hebrew with other filters, write out the chain, keeping `hebrew_lemmatizer` last:

```json
{
  "type": "text",
  "analyzer": {
    "char_filters": ["html_strip"],
    "tokenizer": "unicode_word",
    "filters": ["nfkc_cf", "fold", {"name": "hebrew_lemmatizer", "params": {"suffix": "$"}}]
  }
}
```

## Migrating from Elasticsearch

Mappings that name the Elasticsearch Hebrew plugin's `hebrew` or `hebrew_query` analyzer get the
WarpSearch analyzers of the same names. `hebrew_exact` maps to `hebrew` and `hebrew_query_light`
to `hebrew_query`, each with a warning: search the `$` forms for exact matches, as above. Results match the Java analyzer run over the same
dictionary, which WarpSearch tests against. See
[Through the compatible APIs](text-analysis.md#through-the-compatible-apis).