# Elasticsearch compatibility

WarpSearch answers the Elasticsearch REST API at `https://api.warpsearch.io/compat/es`. Official clients, Logstash, Beats, Fluent Bit and Grafana's Elasticsearch datasource talk to it the same way they talk to a cluster. The root URL reports Elasticsearch 8.11.0 and sends the `X-Elastic-Product: Elasticsearch` header that 8.x clients require.

The rule is simple. If a query or parameter can be translated exactly, it is. If only the scores differ and the same documents match, it runs and a `Warning: 299` response header tells you what was approximated. Anything else fails with `400` and error type `unsupported_feature`, so you never get a silently wrong answer.

## Compatibility at a glance

| Area | Status | Notes |
|---|---|---|
| `_search`, `_msearch`, `_count` | Available | Scores differ from Elasticsearch, see [Behavior differences](#behavior-differences) |
| Full-text and structured queries | Partial | Most query types work, see the [query table](#query-dsl) |
| Aggregations | Partial | Terms, histograms, ranges, metrics, sketches and `top_hits`; no pipeline aggregations |
| kNN and hybrid search | Available | Top-level `knn`, the `knn` query, and `knn` beside a `query` |
| Index create, get, delete, mapping | Available | No dynamic mapping, see [Mappings](#mappings) |
| Single-document APIs and `_bulk` | Partial | No partial updates, conditional writes or real versions |
| Grafana Elasticsearch datasource | Available | Logs and metrics panels |
| Highlighting | Not supported | Coming soon |
| Scroll, `search_after`, point in time | Not supported | |
| Nested documents, joins | Not supported | |
| Scripting (Painless) | Not supported | |
| Searching several indices at once | Not supported | One collection per search |
| Ingest pipelines, templates, aliases, ILM, snapshots | Not supported | |
| Kibana | Not supported | |

## Connect

Create an API key in the [management console](/console/). The role of the key decides what it may do: viewer searches, editor also writes documents, admin also creates collections and changes mappings.

WarpSearch accepts the key in any of these forms:

| Header | Value |
|---|---|
| `Authorization: Bearer` | the key |
| `Authorization: ApiKey` | the key, or the base64 `id:secret` form Elasticsearch clients send |
| `X-API-Key` | the key |

> **Not supported:** HTTP Basic authentication with a console API key as the password. Configure your client to send one of the headers above.

Python:

```python
from elasticsearch import Elasticsearch

es = Elasticsearch(
    "https://api.warpsearch.io/compat/es",
    api_key="ws_live_...",
)

es.search(index="books", query={"match": {"title": "dune"}})
```

curl:

```bash
curl -s https://api.warpsearch.io/compat/es/books/_search \
  -H "Authorization: ApiKey ws_live_..." \
  -H "Content-Type: application/json" \
  -d '{"query": {"match": {"title": "dune"}}}'
```

For the Java clients, set the same `Authorization: ApiKey ...` header as a default header on the REST client. The Java API client sends `typed_keys=true`, which WarpSearch does not honor, so aggregation results do not parse through it. Use the low-level REST client or raw JSON for aggregations.

Logstash and Beats use their normal Elasticsearch output with `hosts` set to the base URL and `api_key` set to your key. Grafana: add an Elasticsearch datasource with the base URL, send the key as a custom header, and choose the collection as the index.

## Endpoints

Index names are collection names.

| Endpoint | Notes |
|---|---|
| `GET /` | Version probe |
| `GET /_security/_authenticate` | Who you are and your role per collection |
| `GET`, `POST /{index}/_search` | Also `q`, `from`, `size` URL parameters |
| `POST /_msearch`, `/{index}/_msearch` | Only the `index` header key is read, and an index list uses its first name |
| `GET`, `POST /{index}/_count` | Exact count |
| `PUT`, `GET`, `HEAD`, `DELETE /{index}` | Create (with mappings and analysis settings), read, exists, delete |
| `GET`, `PUT`, `POST /{index}/_mapping` | Read and add fields |
| `POST /_bulk`, `/{index}/_bulk` | `index`, `create`, `delete`, `update` with `doc_as_upsert` |
| `PUT`, `POST /{index}/_doc/{id}`, `POST /{index}/_doc`, `PUT`, `POST /{index}/_create/{id}` | Index one document |
| `GET`, `HEAD /{index}/_doc/{id}`, `GET /{index}/_source/{id}` | Get one document |
| `DELETE /{index}/_doc/{id}` | Delete one document |
| `GET`, `POST /_mget`, `/{index}/_mget` | Multi-get |
| `POST /_refresh`, `/{index}/_refresh` | Make writes searchable and build approximate vector indexes |
| `POST /_forcemerge`, `/{index}/_forcemerge` | Admin role; use it after a bulk load |

Any other route answers `404`.

## Query DSL

| Query | Support |
|---|---|
| `match`, `match_phrase`, `match_bool_prefix` | Exact, including `operator`, `minimum_should_match`, `fuzziness`, `slop`, `zero_terms_query` |
| `match_phrase_prefix` | One word is exact. Several words are rejected |
| `multi_match` | `best_fields`, `phrase`, `most_fields`, `cross_fields`, `bool_prefix`. Same documents, summed scores |
| `combined_fields` | Exact |
| `term`, `terms`, `ids`, `exists` | Exact. `term` on a text field matches one indexed token |
| `range` | Exact, with `time_zone`, `format`, date math |
| `prefix`, `wildcard`, `regexp`, `fuzzy` | Exact. Regex operators `~ & @ # <n-m>` are rejected |
| `bool` | Exact, every `minimum_should_match` form |
| `constant_score`, `boosting`, `dis_max` | `dis_max` sums clause scores |
| `function_score` | `weight`, `field_value_factor`, decays, `random_score`, filtered functions. No scripts |
| `query_string`, `simple_query_string` | Most options. `simple_query_string` ignores field boosts |
| `geo_distance`, `geo_bounding_box` | Exact |
| `knn` | See [kNN](#knn-and-hybrid-search) |
| `post_filter`, `min_score`, `collapse` with `inner_hits`, `sort` with `missing` and `mode`, `_geo_distance` sort | Exact |

> **Coming soon:** exact scoring for `dis_max`, `multi_match` `best_fields` and field patterns such as `title*`, multi-word `match_phrase_prefix`, and `highlight`. Today `highlight` is accepted with a warning and hits carry no highlight section.

## Aggregations

| Aggregation | Support |
|---|---|
| `terms` | Counts, `size`, `min_doc_count`, `missing`, sub-aggregations. `order` by `_key` or ascending `_count` falls back to count descending with a warning |
| `histogram`, `date_histogram` | `calendar_interval`, `fixed_interval`, `time_zone`, `offset`, `min_doc_count`, `extended_bounds`, `hard_bounds`, `format`, `keyed` |
| `range`, `date_range`, `filters`, `filter`, `global` | Exact |
| `avg`, `sum`, `min`, `max`, `value_count`, `stats`, `extended_stats` | Exact, `field` only |
| `cardinality` | Approximate, as in Elasticsearch. `precision_threshold` honored up to 40000 |
| `percentiles`, `percentile_ranks` | t-digest. `hdr` is rejected |
| `top_hits` | One per bucket aggregation |

Sub-aggregations under a `terms` aggregation need a string or id field. Aggregation names can be any string and come back unchanged.

> **Not supported:** pipeline aggregations (`bucket_script`, `derivative`, `cumulative_sum`, `moving_fn`, and the rest), `composite`, `multi_terms`, `rare_terms`, `significant_terms`, `sampler`, terms `include` and `exclude`, geo aggregations, and any aggregation with a `script` or `missing` on metrics.

## kNN and hybrid search

Send the Elasticsearch top-level `knn` section, or a `knn` query:

```json
{
  "knn": {
    "field": "embedding",
    "query_vector": [0.12, -0.03, 0.88],
    "k": 10,
    "num_candidates": 100,
    "filter": {"term": {"genre": "scifi"}}
  }
}
```

A `knn` section beside a `query` is hybrid search. A document matches if either side matches, and its score is the sum of the two scores times their boosts. Several `knn` sections and `boost` work the same way. A `knn` filter narrows the vector search itself, before ranking.

| Setting | Behavior |
|---|---|
| `k` | Bounded by your plan's result window |
| `num_candidates` | Up to 10000, and at least `k`. It sizes the pool that is rescored at full precision |
| `rescore_vector.oversample` | Widens the rescoring pool |
| `similarity` | Minimum similarity, in the metric's own space |
| Dimensions | Up to 8192 |
| `ivf` (extension) | `{"nprobe": 8}` or `{"min_scan_fraction": 0.1}` to tune recall of approximate indexes |
| `exact` (extension) | `true` forces a full scan, which gives ground truth for measuring recall |

Scores are WarpSearch similarity scores, not Elasticsearch's rescaled `_score`. Rank order is what to compare.

A `dense_vector` mapping with `index_options.type` such as `hnsw` or `int8_hnsw` asks for an approximate index. It is built when you call `_refresh` or `_forcemerge`; until then searches scan exactly. Build parameters such as `m` and `ef_construction` are accepted and ignored with a warning. See [Vector search](vector-search.md) and [Hybrid search](hybrid-search.md).

> **Not supported:** `retriever` and `rank` fusion, `query_vector_builder`, `semantic` and `sparse_vector` queries, `inner_hits` on nested vectors, and `bit` vectors (read as floats).

## Mappings

There is no dynamic mapping. A field in a document must either be mapped or carry a type suffix in its name (`_t` text, `_s` string, `_i` integer, `_f` float, `_d` double, `_dt` date, `_v` vector). Documents with other unmapped fields are rejected. See [Schemas and field types](schemas-and-fields.md).

Mappings can use `text`, `keyword`, `boolean` (stored as keyword), integer types, `float`, `double`, `scaled_float`, `date`, `geo_point` and `dense_vector`. Multi-fields such as `title.keyword` and `copy_to` work. Custom analyzers in `settings.analysis` are translated where possible; a component that cannot be reproduced (stop words, synonyms, n-grams, non-English stemmers) is dropped with a warning.

Field names that are not plain identifiers are mapped for you. `@timestamp` is stored as `timestamp`, and dots become underscores, so `event.created` is stored as `event_created`. You keep using your own names in queries and responses. If two names collide after mapping, the request fails.

> **Not supported:** `nested`, `join`, `ip`, `flattened`, range types, `geo_shape`, `completion`, `binary` and `alias` fields. `boolean` and `ip` values are stored as strings today.

> **Coming soon:** native `boolean` and `ip` types, `flattened`, range types and polygon geo queries.

## Write semantics

`_bulk` batches consecutive actions on one index and reports each item's result separately. Overload is reported per item as `429` so clients can retry just those items. An unknown index is created on first write if your key has the admin role.

- Without `refresh`, new documents become searchable on WarpSearch's own schedule. Use `?refresh=true` or `wait_for` to make them searchable before the call returns.
- `create` and `_create` overwrite an existing document instead of failing with `409`.
- `_version` is always `1`, and `result` is always `created` (or `updated` for an upsert) and `deleted`. A delete of a missing id is not an error.
- `update` works only as `doc_as_upsert` with a whole document, which replaces the stored one. Partial updates fail per item.
- `version`, `if_seq_no`, `if_primary_term` and `routing` are ignored. Document gets see committed documents only.

> **Not supported:** `_update`, `_update_by_query`, `_delete_by_query`, `_reindex`, ingest pipelines (the `pipeline` parameter is ignored), and conditional writes.

## Behavior differences

- **Scores differ.** Relevance scoring is not identical to Elasticsearch. Expect different `_score` values and occasional rank changes. Which documents match is exact wherever a feature is supported.
- **`_source` is rebuilt** from stored fields with your field names and object nesting. Booleans come back as strings and `geo_point` values may be absent.
- **Totals are exact.** `hits.total.relation` is always `eq`, and `track_total_hits` bounds are not applied (`false` still skips the count).
- **Search responses omit `_shards`**, and `timed_out` is always `false`. `typed_keys` and `rest_total_hits_as_int` are not supported.
- **Hits carry no `sort` values**, `_version` or `_seq_no`.
- **Limits apply** to result size, result window, facet size, query size and request rate. See [Limits and errors](limits-and-errors.md).

> **Coming soon:** bounded total hits for faster ranked queries.

## Errors and warnings

An unsupported feature fails with `400` in the Elasticsearch error shape:

```json
{"error": {"type": "unsupported_feature", "reason": "..."}, "status": 400}
```

Unknown query or aggregation names are a normal `query_error`. An approximation returns `200` plus a `Warning: 299` header. Your client library may surface these as warnings; log them while you test.

## Not supported, by theme

- **Paging and exports:** scroll, `search_after`, point in time, slices.
- **Text:** span, `intervals`, `more_like_this`, `terms_set`, `percolate`, suggesters, `rescore`, `search_as_you_type` subfields, `highlight`.
- **Data model:** nested and join fields, scripts and `runtime_mappings`, `script_fields`.
- **Several indices:** index patterns such as `logs-*`, multi-index searches, aliases, data streams.
- **Cluster and lifecycle APIs:** index templates, ILM, snapshots, cross-cluster search, `_cat`, `_cluster`, `_nodes`, `_field_caps`, `_validate`, `_sql`, `_async_search`.
- **Applications:** Kibana, ES|QL.

See [Roadmap and status](roadmap-and-status.md) for what is in development, and [Query languages](query-languages.md) for the native query languages that cover some of these gaps.