For AI agents: this page is also available as Markdown at /docs/more-languages.md. Every docs page is: add .md to its URL. llms.txt lists them all.

More languages

These analyzers follow the chains of Lucene's language analyzers, the ones Elasticsearch ships under the same names, with Lucene's stop words. Their normalizers and stemmers produce the same words as Lucene's, so an index migrated from Elasticsearch matches the same queries.

{
  "fields": {
    "title_fa": {"type": "text", "analyzer": "persian"},
    "title_hi": {"type": "text", "analyzer": "hindi"},
    "title_tr": {"type": "text", "analyzer": "turkish"}
  }
}
Analyzer Language Chain after unicode_word
persian Persian (Farsi) persian_zwnj char filter, lowercase, decimal_digit, arabic_normalize, persian_normalization, stop words, persian_stem
sorani Sorani (Central Kurdish) sorani_normalization, lowercase, decimal_digit, stop words, sorani_stem
hindi Hindi lowercase, decimal_digit, indic_normalization, hindi_normalization, stop words, hindi_stem
bengali Bengali lowercase, decimal_digit, indic_normalization, bengali_normalization, stop words, bengali_stem
telugu Telugu decimal_digit, indic_normalization, telugu_normalization, stop words, telugu_stem
bulgarian Bulgarian lowercase, stop words, bulgarian_stem
latvian Latvian lowercase, stop words, latvian_stem
galician Galician lowercase, stop words, galician_stem
brazilian Brazilian Portuguese lowercase, stop words, brazilian_stem
turkish Turkish apostrophe, turkish_lowercase, stop words, Snowball Turkish

Persian text often joins word parts with a zero-width non-joiner. The persian_zwnj char filter turns it into a space, so می‌خواهم is split the way Persian readers see it, and decimal_digit turns Persian and Arabic-Indic digits into 0 to 9, so ۱۴۰۲ matches 1402.

For Turkish, turkish_lowercase lowercases I to ı and İ to i, and apostrophe cuts the suffix after an apostrophe on proper names, so Türkiye'nin is indexed as türkiye.

Filters

Use these in your own chains. None takes parameters except elision.

Filter Does
elision Removes a leading article and its apostrophe (' or ’): l'avion to avion. articles lists them (default: the French ones), articles_case: true matches them in any case.
apostrophe Cuts a token at its first apostrophe.
decimal_digit Every Unicode decimal digit to its ASCII digit.
indic_normalization Normalizes the Unicode spellings of Indic scripts.
hindi_normalization, bengali_normalization, telugu_normalization Spelling normalization for the language.
persian_normalization, sorani_normalization Spelling normalization for the language.
german_normalization Folds ä, ö, ü and ß the way German readers spell them without umlauts, so Häuser matches Haeuser and Hauser.
turkish_lowercase Turkish case rules for dotted and dotless i.
hindi_stem, bengali_stem, telugu_stem, persian_stem, sorani_stem Light stemmers.
bulgarian_stem, latvian_stem, galician_stem, brazilian_stem Stemmers.

A German chain that matches words written with and without umlauts:

{
  "type": "text",
  "analyzer": {
    "tokenizer": "unicode_word",
    "filters": ["lowercase", "german_normalization", {"name": "snowball", "params": {"language": "german"}}]
  }
}

A French chain with elision and Snowball stemming:

{
  "type": "text",
  "analyzer": {
    "tokenizer": "unicode_word",
    "filters": [{"name": "elision", "params": {"articles": ["l", "d", "qu", "j"], "articles_case": true}},
      "lowercase", {"name": "snowball", "params": {"language": "french"}}]
  }
}

The normalizations and elision also apply to prefix and fuzzy terms, so häu* finds häuser. A wildcard is folded one character at a time, so rules that look at neighbouring characters do not apply to it: hae* does not find häuser, though the prefix query hae does. Stemmers do not apply to prefix, wildcard and fuzzy terms.

Differences from Lucene

Through the compatible APIs

Elasticsearch's persian, sorani, bengali, hindi, bulgarian, latvian, galician, brazilian and turkish analyzers map to the analyzers of the same names. The stemmer filter with these languages, elision, apostrophe, decimal_digit, the normalization filters and lowercase with language: turkish map to the filters above. Solr's text_fa, text_ckb, text_hi, text_bg, text_lv, text_gl and text_tr types and the matching factories and analyzer classes map the same way. See Through the compatible APIs.

View this page as Markdown

Cookie settings

Optional tracking stays off until you choose it. Your choice applies to this browser for 180 days. Read the details.

Essential storage keeps you signed in, protects forms, and remembers this choice. It is always available.