# Schemas and field types

A schema tells WarpSearch how to treat each field: whether to split text into words, keep values
whole, compare numbers, bucket dates or index vectors. You give it a schema in one of two ways, and
you can mix them. Declare fields in an Elasticsearch-style **mapping** when you create the
collection, or let a **field-name suffix** such as `_t` or `_s` choose the type.

There is no dynamic mapping beyond those suffixes. A field that is neither mapped nor
suffix-typed is rejected when you index the document, so a typo cannot silently create a field.

## Field types

| Mapping type | Use it for | Notes |
|---|---|---|
| `text` | Titles, descriptions, article bodies | Split into words, searched with BM25. Cannot be sorted or used for whole-value facets (use a `keyword` sub-field). |
| `keyword` | Categories, tags, codes, ids, anything matched whole | Exact match, filtering, sorting, faceting. Not split into words. |
| `integer`, `long`, `short`, `byte` | Counts, years, quantities | Stored as a signed 64-bit integer. |
| `float`, `half_float` | Measurements | Single precision. |
| `double`, `scaled_float` | Prices, scores, anything needing precision | Double precision. |
| `date` | Timestamps | Millisecond precision. ISO-8601 text or epoch milliseconds. |
| `dense_vector` | Embeddings | Needs `dims` and `similarity` (`cosine`, `l2_norm`, `dot_product`, `max_inner_product`). Up to 8192 dimensions. |
| `knn_vector` | Embeddings, OpenSearch style | Needs `dimension` and `method.space_type` (`l2`, `cosinesimil`, `innerproduct`). |
| `geo_point` | Locations | Send `[longitude, latitude]`. |
| `boolean` | Flags | Stored as text today. |

> **Coming soon:** true `boolean`, `ip` and `decimal` types, the `flattened` type, range field
> types such as `integer_range`, and polygon geo queries. Until they ship, store flags as
> `keyword` values and IPs as `keyword`.

> **Not supported:** `nested` and `join` fields, and arrays of objects. Flatten your data, or
> model relations with ids.

### Text versus keyword

Both accept strings, but they make different things searchable. Index `"Red Bicycle"`:

- In a `text` field it becomes the words `red` and `bicycle`. A search for `bicycle` matches, and
  so does the phrase `"red bicycle"`.
- In a `keyword` field it stays `Red Bicycle`. Only that exact value matches, and you can sort and
  facet on it.

When one value needs both, give the field a `keyword` sub-field. This is the usual Elasticsearch
multi-field:

```json
"title": {
  "type": "text",
  "fields": { "keyword": { "type": "keyword" } }
}
```

You search `title` for words and use `title.keyword` for sorting, exact matches and facets.

## Creating a mapping

Send `mappings` when you create the collection, or add fields later with
`PUT /v1/collections/{c}/mapping`. Both need the `admin` role.

```bash
curl -X PUT https://api.warpsearch.io/v1/collections/products \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mappings": {
      "properties": {
        "name":      {"type": "text", "fields": {"keyword": {"type": "keyword"}}},
        "brand":     {"type": "keyword"},
        "price":     {"type": "double"},
        "released":  {"type": "date"},
        "location":  {"type": "geo_point"},
        "embedding": {"type": "dense_vector", "dims": 768, "similarity": "cosine"}
      }
    }
  }'
```

Object properties are flattened into one field per path, so `meta.year` becomes a single field.
You keep using your own names in documents and queries. Inside WarpSearch the characters `@` and
`.` become `_` and a leading `_` is dropped (`@timestamp` is stored as `timestamp`), and two mapped
names that would collapse into one are rejected. Do not send documents that use the stored name of
a renamed field.

Field names are ASCII letters, digits and `_`, start with a letter, never contain `__` and are at
most 127 bytes after this renaming.

`copy_to` is supported: values of the source field are appended to the target at ingest.

## Name-suffix templates

A field you did not map is typed by the end of its name:

| Suffix | Type |
|---|---|
| `_t` | `text` with word splitting, case and accent folding, and English stemming. The general-purpose text field. |
| `_s` | `keyword`, indexed and sortable. |
| `_ss` | Multi-valued `keyword`. |
| `_i`, `_is` | Integer, single or multi-valued. |
| `_f`, `_fs` | Float, single or multi-valued. |
| `_d`, `_ds` | Double, single or multi-valued. |
| `_dt`, `_dts` | Date, single or multi-valued. |
| `_v`, `_vs` | Vector, single or multi-valued. A vector field needs a mapping with `dims` and `similarity` to be searchable by nearest neighbor. |
| `_name`, `_names` | Names: searchable by word, with the whole value kept for facets and sorting. |

Other suffix variants exist for case-sensitive text (`_u`, `_un`, `_w`, `_wl`) and column-only
strings (`_sc`, `_ssc`). A document like this needs no mapping at all:

```json
{"id": "b1", "title_t": "The Way of Kings", "genre_s": "fantasy", "year_i": 2010,
 "tags_ss": ["epic", "fantasy"], "published_dt": "2010-08-31"}
```

Suffixes are also the answer for fields you cannot list in advance, such as per-category
attributes: any new `something_s` field just works. Geo fields have no suffix and always need a
mapping.

## Analyzers

A `text` field has an analyzer that splits and normalizes text, at index time and at query time.
The default is Unicode word splitting with lowercasing. You can choose:

| `analyzer` | Behavior |
|---|---|
| (default), `standard` | Unicode word segmentation and lowercasing. |
| `whitespace` | Split on whitespace only. |
| `keyword` | The whole value as one token. |
| `english` | Word segmentation, case and accent folding, possessive removal and English stemming (`ponies` matches `pony`). No stop words. |

Custom analyzers go in `settings.analysis` when you create the collection. You can combine the
`standard`, `whitespace` and `keyword` tokenizers with the `lowercase`, `asciifolding`,
`possessive_english` and English stemming filters (`kstem`, `stemmer` with `light_english`) and the
`html_strip` character filter. Anything else in a custom analyzer, such as `stop`, `synonym` or
n-gram filters, is dropped with a warning, so check the `warnings` in the response. Other
built-in analyzers (`simple`, `stop`, other languages) fall back to `standard` with a warning.

> **Coming soon:** `edge_ngram` and `shingle` filters.

> **Not supported:** synonyms, stop words, and language analyzers other than English stemming.

## Dates

A date field takes epoch milliseconds or an ISO-8601 string: `2026-06-25T10:30:00Z`,
`2026-06-25T10:30:00-05:00`, `2026-06-25` or `2026-06`. A date without a zone is read as UTC at
ingest. A partial date in a query names the whole window, so `2026-06` matches all of June.
Queries also accept date math such as `now-30d` and `now/d`, and a request-level `time_zone`.
Use an explicit `Z` or offset in documents when the exact instant matters.

## What you get back

Search hits return the fields you ask for in `fields`. Text and keyword values come back as sent,
and numbers and dates come back from their stored values. Vectors are returned only when you name
them, and geo points are not returned today, so keep coordinates in separate numeric fields if
you need to display them.

> **Coming soon:** a stored `_source` copy of each document.

## Changing a schema

Schema changes apply to documents you write afterwards. They never rewrite or backfill what is
already stored:

- Adding a field is safe. Older documents simply do not have it.
- Redefining an existing field is not checked against existing data. To change a field's type or
  analysis, add a new field, reindex from your source data, and switch your queries over.
- Removing a field and reusing its name does not erase the old indexed values.

Plan the keyword and text split before a large load, because changing it later means reloading.

## Gotchas

- Keyword values longer than 255 bytes are kept distinct, but prefix, wildcard and fuzzy queries
  may not reach the cut-off part. Do not use long values for pattern matching.
- A `text` field cannot be sorted or faceted by whole value. Use a `keyword` sub-field.
- Numbers sent as strings are accepted and converted, and a non-numeric string in a numeric
  field fails that document.
- A multi-valued field takes a JSON array. Text values in an array are separate: a phrase does not
  match across two values.

See also [Ingesting documents](ingesting-documents.md), [Vector search](vector-search.md) and
[Elasticsearch compatibility](compat-elasticsearch.md) for the full list of supported mapping
parameters.