# Writes, visibility and durability

WarpSearch separates accepting a write from making it searchable. Understanding that gap, and the
few rules around it, is all you need to load data safely and to know what a search can and cannot
see.

## Accepted, then committed, then searchable

A write goes through two steps:

1. **Accepted.** WarpSearch validates the documents and records them. The response tells you how
   many were indexed and which failed.
2. **Committed.** A commit publishes a new, immutable view of the collection. Searches see
   documents only after the commit that includes them.

You choose how the commit happens:

| Option | Behavior | Use it when |
|---|---|---|
| `?commit=true` on the write | The request returns only after the commit, so the documents are searchable when you get the response. | You must read your own write: a form submit, a test, the last batch of a load. |
| Do nothing | WarpSearch commits on its own schedule, by size or time, so documents are searchable within seconds. | Streams, logs and bulk loads. |
| `POST /v1/collections/{c}/refresh` | Commits pending writes now. | You loaded without `commit=true` and want to cut over. |

Do not commit after every small write. Each commit publishes a new view, and explicit refreshes
are billed operations; see [Pricing](pricing.md#keeping-write-costs-down). For a bulk load, send the
batches without `commit=true` and commit with the last one.

```bash
curl -X POST "https://api.warpsearch.io/v1/collections/books/documents?commit=true" \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"id": "b1", "title": "Dune", "year": 1965}]'
```

A document fetched by id (`GET /v1/collections/{c}/documents/{id}`) follows the same rule: it is
visible once committed, not instantly.

A search never sees half of a commit. A commit is crash-safe: either the whole new view is
published or the previous one stays.

## Replace is whole-document

A document is identified by its `id`. Writing a document with an existing `id` replaces the old
one completely. Fields you leave out of the new version disappear.

```bash
# Before: {"id": "b1", "title": "Dune", "year": 1965}
curl -X POST "https://api.warpsearch.io/v1/collections/books/documents?commit=true" \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"id": "b1", "title": "Dune"}]'
# After: {"id": "b1", "title": "Dune"}   (year is gone)
```

String and numeric ids are accepted, and `123` and `"123"` are the same id. A document with no
`id` can be searched but cannot be replaced or deleted by id, so always send one.

> **Not supported:** partial or field-level updates, and delete by query. Read the document, change
> it in your application and write the whole document back. To delete many documents, delete them
> by id.

Deleting an id that is not there is a successful no-op, so deletes are safe to repeat.

## Retrying safely

Because a write with an `id` replaces the previous version, retrying the same batch is harmless:
you end up with one copy of each document. Make your loader retry-friendly:

- Always send stable ids, derived from your source data.
- Read the response body. A request can return `200` and still list per-document failures, and
  `207` when part of a large request was not written; the `errors` list gives each failed
  document's `index` and `id`. Retry exactly those. See
  [Ingesting documents](ingesting-documents.md#partial-failures).
- After a timeout or a dropped connection, assume the write may have been applied and send it
  again. WarpSearch does not retry writes for you, so a timeout can mean either outcome.
- On `429` (overload) or `503 warming_up`, wait for `Retry-After` and retry.
- Use `all_or_none=true` when a batch must apply entirely or not at all. It covers one request
  and must fit in a single backend update, so keep such batches small.

If you pass `allow_dups` through a compatible API (for example Solr's `overwrite=false`) to speed
up a load, retries can create duplicates. Leave it off unless every id is new.

## Durability

WarpSearch keeps collection data in durable storage, and the compute that serves a collection is
disposable. A commit makes your documents durable and visible, and WarpSearch copies committed
changes to durable storage shortly afterwards. An idle collection is saved there too and reloaded on
its next request.

What this means in practice:

- A write is guaranteed only after its commit. Writes accepted but not yet committed can be lost
  if the infrastructure serving the collection fails at that moment.
- A very recent commit may not have reached durable storage yet when such a failure happens, so
  the newest minutes of a heavy load are the exposed ones.
- Do not treat WarpSearch as your system of record while a load is in flight. Keep the source
  data until the load has finished and a count or a sample search confirms the documents are
  there, and load with stable ids so you can replay.
- Collection deletion is permanent. There is no undo and no backup you can restore from yet, so
  keep your source of truth and reload by replaying it.

Schema changes follow the same model: they apply to documents written after the change and do not
rewrite old ones. See [Schemas and field types](schemas-and-fields.md#changing-a-schema).

## Reads see one consistent view

A single search request, including its hits, facets and metrics, is computed over one consistent
view of the collection, so a facet count never disagrees with the hit total because a commit
landed in between. Two separate requests can see different views if a commit lands between them.

## Cold starts

A collection with no recent requests is unloaded and reloaded on demand. The first request after
that can answer `503 warming_up` with `Retry-After`. No data is lost and nothing is billed for the
wait. See [Collections](collections.md#idle-collections-and-cold-starts).

## Quick reference

| Question | Answer |
|---|---|
| When is a write searchable? | After its commit: immediately with `commit=true`, otherwise within seconds. |
| Does a replace merge fields? | No. The new document replaces the old one entirely. |
| Can I update one field? | No. Write the whole document again. |
| Can I delete by query? | No. Delete by id. |
| Is retrying a write safe? | Yes, when documents carry stable ids. |
| What do I do on `503 warming_up`? | Retry after `Retry-After`. |