# Loki and LogQL

WarpSearch understands LogQL as a query language. It does not implement the Loki HTTP API. Those are two different things, and this page keeps them apart so you know what will work before you start.

| You want to | Status |
|---|---|
| Run LogQL queries against logs stored in WarpSearch | Available, a practical subset |
| Send logs with Promtail, Grafana Alloy `loki.write`, Fluent Bit `loki` output or the Docker Loki driver | Not supported |
| Query from Grafana's Loki datasource | Not supported |
| Send logs with Logstash, Beats, Fluent Bit, Vector or Data Prepper through their Elasticsearch or OpenSearch outputs | Available, test your shipper |
| Query from Grafana's Elasticsearch or OpenSearch datasource | Available |

> **Not supported:** the Loki HTTP API. `/loki/api/v1/push`, `/loki/api/v1/query` and `/loki/api/v1/query_range` do not exist, so Promtail, Grafana Alloy's `loki.write`, Fluent Bit's `loki` output and Grafana's Loki datasource cannot be pointed at WarpSearch today. OTLP and Prometheus remote-write ingest are not supported either.

## Run LogQL queries

LogQL is one of the languages the search API accepts. Set `language` to `logql`. You can use the REST API, or the `search` tool of the [MCP server](agents.md).

```bash
curl -s https://api.warpsearch.io/v1/search \
  -H "Authorization: Bearer ws_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "collection": "logs",
    "language": "logql",
    "query": "{app=\"api\", level=~\"warn|error\"} |= \"timeout\"",
    "time_range": {"from": "now-1h", "to": "now"},
    "limit": 100
  }'
```

`time_range` takes ISO-8601 times, epoch milliseconds or `now` with an offset such as `now-15m`. Log queries return matching documents, newest first, 100 by default. Metric queries return a date histogram instead of documents; use `step` to set the bucket width:

```json
{
  "collection": "logs",
  "language": "logql",
  "query": "sum by (host) (count_over_time({app=\"api\"} |= \"error\" [5m]))",
  "time_range": {"from": "now-6h", "to": "now"},
  "step": "1m"
}
```

To see how a query is interpreted without running it, call `POST /v1/translate` with the same body.

### Labels are fields

In Loki, labels describe a stream. In WarpSearch there are no streams. Every label is just a field on the document, and a stream selector is a set of field filters.

| LogQL | What WarpSearch does |
|---|---|
| `{app="api"}` | Exact match on the `app` field |
| `{app!="api"}` | Exclude documents where `app` equals `api` |
| `{level=~"warn\|error"}` | Match either value. Plain alternations are an exact membership test, which is fast |
| `{host!~"web.*"}` | Regular expression against whole values |
| `{app=~".+"}` | The field exists |

A selector needs at least one matcher.

### Line filters

| LogQL | What WarpSearch does |
|---|---|
| `\|= "timeout"` | The words or phrase appear in the message field |
| `!= "healthcheck"` | They do not appear |
| `\|~ "err.*"` | The regular expression matches a whole indexed word, with a warning |
| `!~ "..."` | The regular expression matches no indexed word |
| `\|= "a" or "b"` | Either alternative |

Line filters search analyzed words, not raw text. `|= "time"` does not match the word `timeout`, and punctuation is not part of what is matched. The exact matching rules (case, accents, stemming) follow the analyzer of the message field. Regular expression filters match whole indexed words, so WarpSearch widens them to match anywhere inside a word, and they may be disabled for your account. If you need a raw substring match, put the value in its own string field and filter on that field.

### Parser stages and label filters

| LogQL | What WarpSearch does |
|---|---|
| `\| json`, `\| logfmt`, `\| unpack` | Accepted and ignored. Your fields were already extracted when you indexed the document |
| `\| status >= 500`, `\| level="error"` | Filters on the `status` or `level` field. Numbers, durations and strings, with `and`, `or` and commas |
| `\| unwrap latency_ms` | Chooses the numeric field for `sum_over_time`, `avg_over_time`, `min_over_time` and `max_over_time` |
| `\| drop`, `\| keep` | Accepted and ignored |
| `\| pattern`, `\| regexp`, `\| line_format`, `\| label_format` | Accepted and ignored, with a warning |

Because parsing happens when you ingest, a field that Loki would extract at query time (for example `status` from a JSON line) has to be a real field on the document for the label filter to work.

### Metric queries

| LogQL | What WarpSearch does |
|---|---|
| `count_over_time(... [5m])` | Document counts per time bucket |
| `rate(... [5m])` | The same counts. Divide by the bucket width for a per-second rate. A warning says so |
| `sum_over_time`, `avg_over_time`, `min_over_time`, `max_over_time` with `unwrap` | The metric of the numeric field per bucket |
| `sum`, `count`, `avg`, `min`, `max` with `by (labels)` | One series per value of the grouped fields (the top 10 per level) |
| `topk(n, ...)` | The top n series |
| `bottomk(n, ...)` | Approximated by the top series by count, with a warning |

`avg`, `min` and `max` across series are returned per series, with a warning.

> **Not supported:** `without (...)` grouping, `bytes_over_time` and `bytes_rate`, binary operations between queries, `offset`, `quantile_over_time`, `absent_over_time`, `label_replace`, and the `/loki/api/v1/*` response format. Queries that use them return a clear error.

### Requirements on the collection

Line filters search the collection's message field, and metric queries and `time_range` use its time field. Both are settings of the collection. Read them from `GET /v1/collections/{collection}` (the `message_field` and `time_field` properties). If one is empty, a query that needs it fails with an error naming the missing setting. Label selectors and label filters work without either. A `time_range` can name its own `field` instead.

## Send logs to WarpSearch

There is no Loki-compatible push endpoint, so send logs through the Elasticsearch-compatible or OpenSearch-compatible ingest API, or the native documents API.

### Through a log shipper

Logstash, Filebeat, Fluent Bit, Vector and Data Prepper can write to `/compat/es` or `/compat/opensearch`. Use the shipper's Elasticsearch or OpenSearch output, point `hosts` at the base URL and send your API key in an `Authorization: Bearer` or `ApiKey` header. See [Elasticsearch compatibility](compat-elasticsearch.md) and [OpenSearch compatibility](compat-opensearch.md).

Shippers do more than send documents. Many probe the server version, licence or installed templates when they start, and may try to create an index template or an ingest pipeline. WarpSearch serves the version probe and `_bulk`, but it does not serve `_license`, `_xpack`, index templates or ingest pipelines. Whether a given shipper and version works depends on those probes, and we have not verified each one. Test with a small batch first, turn off template and pipeline management in the shipper, and create the collection yourself before the first write.

> **Not supported:** index templates, ingest pipelines, data streams and ILM. A shipper that requires them will not run unchanged.

### Through the documents API

You can also post NDJSON directly:

```bash
curl -s "https://api.warpsearch.io/v1/collections/logs/documents?commit=true" \
  -H "Authorization: Bearer ws_live_..." \
  -H "Content-Type: application/x-ndjson" \
  --data-binary @batch.ndjson
```

A request is also accepted as a JSON array. Requests are limited in documents and bytes by your plan; send logs in batches. See [Ingesting documents](ingesting-documents.md) and [Limits and errors](limits-and-errors.md).

## A schema for logs

Pick field names so the types are clear without a mapping. WarpSearch infers the type of a field from its name suffix, so a log collection can start with no setup:

| Field | Suffix | Type | Use |
|---|---|---|---|
| `timestamp_dt` | `_dt` | date | The time field. Time ranges, histograms and sorting |
| `message_t` | `_t` | text | The message field. Full-text and line filters |
| `app_s`, `level_s`, `host_s` | `_s` | string | Labels. Exact match, `by (...)` grouping, facets |
| `status_i`, `latency_ms_f` | `_i`, `_f` | number | Label filters and `unwrap` |

Keep labels low in cardinality and keep free text in the message field. Fields you will filter on should be separate fields, not buried in the message. If you ship through an Elasticsearch-style output, you can instead create the collection with an explicit mapping and keep `@timestamp` and `message`; WarpSearch stores `@timestamp` as `timestamp`. See [Schemas and field types](schemas-and-fields.md) and [Ingesting documents](ingesting-documents.md).

LogQL line filters and metric queries use the collection's message and time fields, see [Requirements on the collection](#requirements-on-the-collection). Check that the collection description shows them before you rely on those features.

## Query logs from Grafana

Use Grafana's Elasticsearch or OpenSearch datasource, not the Loki datasource. Point it at `https://api.warpsearch.io/compat/es` or `/compat/opensearch`, send your key as a custom header, choose the collection as the index and your date field as the time field, and use the Logs or Metrics query types. Queries there are Lucene syntax, for example `app:api AND level:(warn OR error) AND message:timeout`. See [Query languages](query-languages.md).

> **Not supported:** the Loki datasource, Explore's Loki log volume and context features, and LogQL entered in a Loki-typed panel.

> **Coming soon:** highlighting of matched terms in log views. Hits carry no highlight section today.

See [Roadmap and status](roadmap-and-status.md) for what is in development.