For AI agents: this page is also available as Markdown at /docs/multi-collection-queries.md. Every docs page is: add .md to its URL. llms.txt lists them all.

Multi-collection queries

Use POST /v1/query to combine data from several collections into one result table. You can filter and sort the combined rows, count events across log collections, or join events to an asset inventory. Every collection in the request must be readable by your API key.

Choose either a structured JSON request for filtering and sorting, or PPL for a sequence of operations such as aggregation, joins and JSON extraction. Both return the same table format.

#Query several collections

Suppose app_logs and audit_logs both contain id, host_s and status_i. This request returns the 20 rows with the highest status codes across both collections:

curl -X POST https://api.warpsearch.io/v1/query \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  --data @- <<'EOF_REQUEST'
{
  "collections": ["app_logs", "audit_logs"],
  "fields": ["id", "host_s", "status_i"],
  "where": "status_i >= $minimum",
  "params": {"minimum": 400},
  "sort": [{"field": "status_i", "desc": true}, {"field": "id"}],
  "limit": 20
}
EOF_REQUEST

The sort and limit apply to the combined result. limit: 20 means at most 20 rows overall, not 20 per collection. The inputs are appended without deduplication: equal IDs in different collections remain separate rows.

Use explicit collection names. Collection wildcards and index patterns are not accepted. Fields must have compatible types across the inputs; a field missing from an individual document produces null. Use a PPL KEEP stage in each input to align differently named fields before combining them.

#Structured request fields

where uses PPL syntax, not Lucene, SQL or Elasticsearch Query DSL. Bind variable values through params instead of constructing query text. For example, a supplied string containing | is still a string value, not another stage.

For indexed filtering, aggregation, offsets or joins, use the PPL form below. Resource options in Limits and large inputs apply to both forms.

#Aggregate across collections

The PPL form replaces collections, where, sort and limit with a single query string:

{
  "query": "FROM app_logs, audit_logs | SEARCH term(action_s, $action) | STATS count() AS events BY host_s | SORT events DESC | LIMIT 20",
  "params": {"action": "login"}
}

This counts all matching login events from both collections, groups by host, and returns the 20 hosts with the largest counts. A successful response contains complete counts for the selected inputs, even when the final table contains only 20 groups. If an input or intermediate result exceeds a work limit, the request fails instead of returning counts from a truncated sample.

Put indexed SEARCH stages directly after FROM to narrow large collections before further processing. A later WHERE filters rows already read, and a final LIMIT controls output rather than the amount of input an aggregate needs.

Do not combine query with collections, where, sort or limit in the same body. PPL is sent to /v1/query; it is not a language value for /v1/search, /v1/translate or the existing MCP search tools.

#Join events to another collection

A left join can attach an asset owner to each log event while retaining events with no known owner:

{
  "query": "FROM app_logs AS e | SEARCH term(e.action_s, $action) | JOIN left [FROM assets AS a] ON e.host_s = a.host_s | KEEP e.id AS event_id, e.host_s AS host, a.owner_s AS owner | LIMIT 20",
  "params": {"action": "login"}
}

Use aliases such as e and a to distinguish fields. A host missing from assets has a null owner. If the inventory has two matching rows for a host, each event for that host produces two joined rows. Null keys never join to each other. See PPL joins for inner, semi and anti joins.

#Read the result table

A response has complete: true, a columns array describing the table, and rows containing values in that column order. For example, the aggregation above could return this table (additional execution metadata is omitted):

{
  "version": 1,
  "complete": true,
  "columns": [
    {"name": "events", "type": "int64", "nullable": true},
    {"name": "host_s", "type": "string", "nullable": true}
  ],
  "rows": [
    [42, "web-1"],
    [17, "worker-2"]
  ]
}

Read column names from columns; they are sorted by name, so their order may differ from your KEEP or fields list. This endpoint returns rows, not the hits, facets and metrics shape of /v1/search.

Empty results retain known column names. Their type can be null when no value establishes a type. Mixed, incompatible values in one output column produce an error instead of being silently converted. Use a JSON decoder that preserves 64-bit integers if you return large IDs or exact counts; ordinary JavaScript numbers cannot represent every int64 value exactly.

#Track the source of each row

IDs are scoped to a collection. If you need to distinguish equal IDs in different collections, add your own source column using PPL:

FROM app_logs
| KEEP id, host_s
| EVAL source = "app_logs"
| UNION ALL [FROM audit_logs | KEEP id, host_s | EVAL source = "audit_logs"]
| SORT source, id
| LIMIT 20

There is no automatic source column. UNION ALL preserves duplicates and can also combine separately filtered inputs.

#Limits and large inputs

The current query API processes bounded inputs. Its work limits apply to source rows and intermediate results, including the rows produced by joins and array expansion. Reducing the final limit alone does not reduce those inputs.

Up to 64 sources are accepted, including nested join and union inputs. Query size, expression complexity, output rows and offset windows also obey your plan limits. Use an explicit final LIMIT in PPL or limit in the structured form; without one, the requested work-row budget must also fit the plan's output-row limit.

When a request exceeds a work limit:

1. Add selective SEARCH term(...) or SEARCH match(...) stages immediately after FROM in each PPL input.
2. Request only the fields you need. Use KEEP to make the desired output clear.
3. Check join multiplicity: repeated keys can produce many more rows than either input.
4. Increase a work limit only if your plan permits it and the input is still bounded.

There is no automatic disk spilling, streaming result cursor or unlimited aggregation in this version.

#Access, consistency and errors

Your key needs read access to every named collection, including those inside joins and unions. Field restrictions and collection row filters apply to every input. A collection you cannot see returns 404 rather than contributing an empty input. Configured sources must belong to the same storage cluster.

Each collection is read from its own snapshot. The result is not a transaction across collections: concurrent writes can become visible at different times. A source that cannot be read fails the whole query; no successful partial table is returned. Retry a transient read failure rather than treating it as an empty collection.

Malformed expressions or incompatible values return an error. Request size and plan limits are checked before execution; a join or aggregation that exceeds its runtime budget also fails. See Limits and errors for the common response format and retry guidance.

For the full stage reference and log-analysis recipes, continue with PPL.

View this page as Markdown