# Ingesting documents

You get documents into a collection by posting JSON to it. You can use the native REST API shown
here, the Elasticsearch-compatible `_bulk` API, Solr's update API, the MCP `index_documents` tool,
or the **Ingest** tab of a collection in the [console](/console/). The collection must already
exist; see [Collections](collections.md).

## Document shape

A document is a flat JSON object. Give every document an `id` so you can replace or delete it later.

```json
{
  "id": "b1",
  "title": "The Way of Kings",
  "genre": "fantasy",
  "year": 2010,
  "tags": ["epic", "fantasy"],
  "location": [-74.006, 40.7128],
  "embedding": [0.12, -0.08, 0.31]
}
```

- Field names must be mapped or end in a type suffix such as `_t`, `_s` or `_i`. Anything else is
  rejected for that document. See [Schemas and field types](schemas-and-fields.md).
- Arrays hold multiple values for one field. A multi-valued geo field is an array of
  `[lon, lat]` pairs.
- Geo points are `[longitude, latitude]`, in that order.
- Send vectors yourself, as arrays of numbers with the collection's `dims`. WarpSearch does not
  create embeddings from text. See [Vector search](vector-search.md).
- Nested objects and arrays of objects are rejected. Flatten them first.
- Elasticsearch-style names with `@` or `.` work as long as you mapped them; see
  [Schemas and field types](schemas-and-fields.md).

> **Not supported:** automatic embedding on ingest. Compute embeddings in your pipeline and send
> them in the document.

## Send documents

`POST /v1/collections/{c}/documents` takes a JSON array, an object with a `docs` array, or NDJSON
with one document per line.

```bash
curl -X POST "https://api.warpsearch.io/v1/collections/books/documents?commit=true" \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[
    {"id": "b1", "title": "Dune", "genre": "scifi", "year": 1965},
    {"id": "b2", "title": "Dune Messiah", "genre": "scifi", "year": 1969}
  ]'
```

Options can also go inside an object body:

```json
{"docs": [{"id": "b1", "title": "Dune"}], "commit": true, "all_or_none": false}
```

For a file you already have, send it as NDJSON and let curl pass the bytes through untouched:

```bash
curl -X POST "https://api.warpsearch.io/v1/collections/books/documents" \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary @books.ndjson
```

| Option | Meaning |
|---|---|
| `commit=true` | Make the documents searchable before responding. See [Writes, visibility and durability](consistency-and-durability.md). |
| `all_or_none=true` | Apply the whole request or nothing: one bad document rolls back the rest. The request must fit in a single backend update, so keep these batches small. |

The response reports the outcome:

```json
{"status": "ok", "indexed": 2, "failed": 0, "took_ms": 14}
```

`status` is `ok` (everything written), `partial` (some failed) or `error` (nothing written).

## Replace a document

Posting a document with an `id` that exists replaces the whole document. Fields you leave out are
gone afterwards. There are no partial updates, and the same call both creates and replaces.

> **Not supported:** partial or field-level updates, and delete by query. To change one field,
> read the document, change it and write it back in full.

## Get a document

```bash
curl https://api.warpsearch.io/v1/collections/books/documents/b1 \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY"
```

This returns `{"id": ..., "fields": {...}}`. A document is readable once committed, not in real
time, and it needs only the `viewer` role.

## Delete documents

```bash
# One document
curl -X DELETE "https://api.warpsearch.io/v1/collections/books/documents/b1?commit=true" \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY"

# Many, by id
curl -X POST https://api.warpsearch.io/v1/collections/books/documents/_delete \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"ids": ["b1", "b2"], "commit": true}'
```

Deleting an id that does not exist succeeds. Deleting needs the `editor` role.

## Partial failures

A write can succeed for most documents and fail for a few, and that is still a normal HTTP
response. Always read the body. The failures are listed with the position and id of each:

```json
{
  "status": "partial",
  "indexed": 1,
  "failed": 1,
  "errors": [
    {"index": 1, "id": "b2", "code": "unknown_field", "message": "Field not found: nosuffix"}
  ],
  "took_ms": 9
}
```

A failed document leaves any older version with the same id untouched. Fix the documents named in
`errors` and send just those again.

Large requests are split into several backend updates. If one fails after the first has been
applied, the response is `207` and lists every remaining document as failed, with a warning that
they were not written (and not committed, if you asked for a commit). Retry exactly those
documents. Because writes replace by id, retrying is safe.

Other failures are not per document and are explained in [Limits and errors](limits-and-errors.md):

| Status | Code | Meaning |
|---|---|---|
| `400` | `invalid_request` | Malformed body, or no documents. |
| `403` | `forbidden` | Your key has the `viewer` role, and writing needs `editor`. |
| `404` | `not_found` | The collection does not exist, or your key has no role on it. |
| `413` | `request_too_large` | Over your plan's documents or bytes per request. Send smaller batches. |
| `429` | `rate_limited`, `resource_exhausted` | Slow down and retry after `Retry-After`. |
| `503` | `warming_up` | The collection is loading from idle. Retry after `Retry-After`. |

## Batching and throughput

- Batch your writes. Every write request bills a minimum size, so thousands of one-document
  requests cost far more than a few large ones. See
  [Pricing](pricing.md#keeping-write-costs-down) for the current guidance on batch size.
- Send batches from a few parallel workers, and honor `Retry-After` when you are throttled.
- Do not use `commit=true` on every batch of a bulk load. Commit on the last batch, or let
  WarpSearch commit on its own.
- Large documents (several KB each) produce many small segments on a big load; a single
  `_forcemerge` through the Elasticsearch-compatible API after the load keeps refreshes fast.
- Your plan sets the maximum documents and bytes per request. The limits are in the console and
  are described in [Limits and errors](limits-and-errors.md).

## Other ways in

- **Elasticsearch and OpenSearch.** Use `_bulk` or the single-document `_doc` APIs at
  `/compat/es` or `/compat/opensearch`. Unknown indices are created on the fly when your key is
  allowed to create collections. `update` actions are full replacements; partial updates are
  per-item errors. See [Elasticsearch compatibility](compat-elasticsearch.md) and
  [OpenSearch compatibility](compat-opensearch.md).
- **Solr.** `/compat/solr/{collection}/update` accepts JSON documents and `add`, `delete` by id and
  `commit` commands. Delete by query and atomic updates are not supported. See
  [Solr compatibility](compat-solr.md).
- **AI agents.** The MCP `index_documents` and `delete_documents` tools take the same shapes,
  with `commit: true` for immediate searchability. See [Guide for AI agents](agents.md).

## Checklist

1. Create the collection and its mapping first.
2. Use stable ids so loads can be repeated.
3. Send batches, read every response body, retry only the failed documents.
4. Commit on the last batch, then confirm with a count:

```bash
curl -X POST https://api.warpsearch.io/v1/collections/books/count \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"language": "lucene", "query": "*:*"}'
```