# More languages

These analyzers follow the chains of Lucene's language analyzers, the ones Elasticsearch ships
under the same names, with Lucene's stop words. Their normalizers and stemmers produce the same
words as Lucene's, so an index migrated from Elasticsearch matches the same queries.

```json
{
  "fields": {
    "title_fa": {"type": "text", "analyzer": "persian"},
    "title_hi": {"type": "text", "analyzer": "hindi"},
    "title_tr": {"type": "text", "analyzer": "turkish"}
  }
}
```

| Analyzer | Language | Chain after `unicode_word` |
|---|---|---|
| `persian` | Persian (Farsi) | `persian_zwnj` char filter, `lowercase`, `decimal_digit`, `arabic_normalize`, `persian_normalization`, stop words, `persian_stem` |
| `sorani` | Sorani (Central Kurdish) | `sorani_normalization`, `lowercase`, `decimal_digit`, stop words, `sorani_stem` |
| `hindi` | Hindi | `lowercase`, `decimal_digit`, `indic_normalization`, `hindi_normalization`, stop words, `hindi_stem` |
| `bengali` | Bengali | `lowercase`, `decimal_digit`, `indic_normalization`, `bengali_normalization`, stop words, `bengali_stem` |
| `telugu` | Telugu | `decimal_digit`, `indic_normalization`, `telugu_normalization`, stop words, `telugu_stem` |
| `bulgarian` | Bulgarian | `lowercase`, stop words, `bulgarian_stem` |
| `latvian` | Latvian | `lowercase`, stop words, `latvian_stem` |
| `galician` | Galician | `lowercase`, stop words, `galician_stem` |
| `brazilian` | Brazilian Portuguese | `lowercase`, stop words, `brazilian_stem` |
| `turkish` | Turkish | `apostrophe`, `turkish_lowercase`, stop words, Snowball Turkish |

Persian text often joins word parts with a zero-width non-joiner. The `persian_zwnj` char filter
turns it into a space, so `می‌خواهم` is split the way Persian readers see it, and `decimal_digit`
turns Persian and Arabic-Indic digits into `0` to `9`, so `۱۴۰۲` matches `1402`.

For Turkish, `turkish_lowercase` lowercases `I` to `ı` and `İ` to `i`, and `apostrophe` cuts the
suffix after an apostrophe on proper names, so `Türkiye'nin` is indexed as `türkiye`.

## Filters

Use these in your own chains. None takes parameters except `elision`.

| Filter | Does |
|---|---|
| `elision` | Removes a leading article and its apostrophe (`'` or `’`): `l'avion` to `avion`. `articles` lists them (default: the French ones), `articles_case: true` matches them in any case. |
| `apostrophe` | Cuts a token at its first apostrophe. |
| `decimal_digit` | Every Unicode decimal digit to its ASCII digit. |
| `indic_normalization` | Normalizes the Unicode spellings of Indic scripts. |
| `hindi_normalization`, `bengali_normalization`, `telugu_normalization` | Spelling normalization for the language. |
| `persian_normalization`, `sorani_normalization` | Spelling normalization for the language. |
| `german_normalization` | Folds `ä`, `ö`, `ü` and `ß` the way German readers spell them without umlauts, so `Häuser` matches `Haeuser` and `Hauser`. |
| `turkish_lowercase` | Turkish case rules for dotted and dotless `i`. |
| `hindi_stem`, `bengali_stem`, `telugu_stem`, `persian_stem`, `sorani_stem` | Light stemmers. |
| `bulgarian_stem`, `latvian_stem`, `galician_stem`, `brazilian_stem` | Stemmers. |

A German chain that matches words written with and without umlauts:

```json
{
  "type": "text",
  "analyzer": {
    "tokenizer": "unicode_word",
    "filters": ["lowercase", "german_normalization", {"name": "snowball", "params": {"language": "german"}}]
  }
}
```

A French chain with elision and Snowball stemming:

```json
{
  "type": "text",
  "analyzer": {
    "tokenizer": "unicode_word",
    "filters": [{"name": "elision", "params": {"articles": ["l", "d", "qu", "j"], "articles_case": true}},
      "lowercase", {"name": "snowball", "params": {"language": "french"}}]
  }
}
```

The normalizations and `elision` also apply to prefix and fuzzy terms, so `häu*` finds `häuser`.
A wildcard is folded one character at a time, so rules that look at neighbouring characters do
not apply to it: `hae*` does not find `häuser`, though the prefix query `hae` does. Stemmers do not
apply to prefix, wildcard and fuzzy terms.

## Differences from Lucene

- A token that a filter empties is dropped, where Lucene indexes an empty term.
- In your own chains, the `stop` filter passes its words through the filters before it, so an
  entry the chain would rewrite still matches; Lucene's does not. The analyzers above remove the
  same words as Lucene's.
- The `turkish` analyzer stems with Snowball 3.2, where Lucene bundles an older Snowball.

## Through the compatible APIs

Elasticsearch's `persian`, `sorani`, `bengali`, `hindi`, `bulgarian`, `latvian`, `galician`,
`brazilian` and `turkish` analyzers map to the analyzers of the same names. The `stemmer` filter
with these languages, `elision`, `apostrophe`, `decimal_digit`, the normalization filters and
`lowercase` with `language: turkish` map to the filters above. Solr's `text_fa`, `text_ckb`,
`text_hi`, `text_bg`, `text_lv`, `text_gl` and `text_tr` types and the matching factories and
analyzer classes map the same way. See
[Through the compatible APIs](text-analysis.md#through-the-compatible-apis).
