# Term search vs grep search

Term search and grep search process your query differently. Understanding these differences helps you choose the right search type and avoid unexpected results.

## How they work

### Term search

Term search breaks your query into **tokens** and looks up each token in the search index.

- Dots and underscores do **not** break tokens — `com.example.service` and `connection_timeout` are each treated as a single token.
- Hyphens, colons, slashes, equals signs, brackets, and other punctuation **do** break tokens — `wire-format` becomes two tokens: `wire` and `format`.
- Tokens are lowercased automatically.
- Common English words (stop words) such as `is`, `not`, `in`, `the`, `a`, `and`, `or`, `to` are removed from the query.
- A log matches only if **all** remaining tokens exist in it.
- Token order does not matter.

### Grep search

Grep search looks for the **exact character sequence** you typed, anywhere in the raw log line.

- No tokenization — the entire search string is matched as-is.
- Character order, spacing, and punctuation all matter.
- Case-sensitive — `error` does not match `ERROR`.

## How special characters affect term search

The following table shows how term search tokenizes various inputs.

| Input | Tokens produced |
| --- | --- |
| `user.name` | `user.name` (single token — dot kept) |
| `connection_timeout` | `connection_timeout` (single token — underscore kept) |
| `com.example.service.MyClass` | `com.example.service.myclass` (single token — dots kept) |
| `192.168.1.1` | `192.168.1.1` (single token) |
| `status_code=500` | `status_code`, `500` (equals sign splits) |
| `GET /api/v2/users` | `get`, `api`, `v2`, `users` (slash splits, lowercased) |
| `error-404` | `error`, `404` (hyphen splits) |
| `host:8080` | `host`, `8080` (colon splits) |
| `key=value` | `key`, `value` (equals sign splits) |
| `[ERROR]` | `error` (brackets stripped, lowercased) |
| `error in module` | `error`, `module` (`in` is a stop word — removed) |
| `not found` | `found` (`not` is a stop word — removed) |
| `10:30:00` | `10`, `30`, `00` (colon splits) |
| `2024-01-15T10:30:00Z` | `2024`, `01`, `15t10`, `30`, `00z` (hyphens and colons split; `T` and `Z` stay attached to adjacent digits) |
| `failed: connection_timeout` | `failed`, `connection_timeout` |

### Stop words

The following words are removed from term search queries:

`a`, `an`, `and`, `are`, `as`, `at`, `be`, `but`, `by`, `for`, `if`, `in`, `into`, `is`, `it`, `no`, `not`, `of`, `on`, `or`, `such`, `that`, `the`, `their`, `then`, `than`, `there`, `these`, `they`, `this`, `to`, `was`, `will`, `with`, `those`

## Examples

Each example below shows a full log line, the tokens produced by term search indexing, and then how various searches behave with both term and grep.

### Log line: `cannot parse invalid wire-format data`

Indexed tokens: `cannot`, `parse`, `invalid`, `wire`, `format`, `data`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `cannot parse` | ✅ | ✅ | Tokens `cannot`, `parse` both exist; exact substring exists |
| `parse cannot` | ✅ | ❌ | Token order doesn’t matter for term; but no literal `parse cannot` in log |
| `wire-format` | ✅ | ✅ | Tokens `wire`, `format` (hyphen splits); exact substring exists |
| `wire format` | ✅ | ❌ | Same tokens; but no literal `wire format` in log (log has `wire-format`) |
| `cannot pars` | ❌ | ❌ | `pars` is not a complete token; not an exact substring either |
| `annot pars` | ❌ | ✅ | No tokens `annot` or `pars`; but grep finds the exact substring `annot pars` within `cannot parse` |

### Log line: `com.example.service.scheduler.QuartzJob started`

Indexed tokens: `com.example.service.scheduler.quartzjob`, `started`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `com.example.service.scheduler.QuartzJob` | ✅ | ✅ | Full token match (lowercased); exact substring exists |
| `com.example.service.scheduler` | ❌ | ✅ | Term search requires an exact token match; partial matches (even prefixes of a token) are not supported. Grep finds this substring |
| `scheduler` | ❌ | ✅ | Not a standalone token — it is part of the larger dot-separated token. Grep finds substring |
| `QuartzJob` | ❌ | ✅ | No such token (indexed lowercased as part of larger token). Grep finds substring |
| `started` | ✅ | ✅ | Exact token match; exact substring exists |

### Log line: `invalid data in the message`

Indexed tokens: `invalid`, `data`, `message`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `data in the message` | ✅ | ✅ | `in` and `the` removed → searches `data`, `message`, both match; exact substring exists |
| `data message` | ✅ | ❌ | Same tokens after stop word removal → same term results; but no literal `data message` in log |
| `is not` | ❌ | ❌ | Both words are stop words → empty query → no match; no substring either |

### Log line: `info sbr/skill_based_routing_controller.go:1920 GetChannelList`

Indexed tokens: `info`, `sbr`, `skill_based_routing_controller.go`, `1920`, `getchannellist`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `skill_based_routing_controller.go` | ✅ | ✅ | Full token match (dots and underscores kept); exact substring exists |
| `skill_based` | ❌ | ✅ | Not the full token. Grep finds substring |
| `routing` | ❌ | ✅ | Not a standalone token. Grep finds substring |

### Log line: `2024-01-15T10:30:00Z [ERROR] GET /api/v2/users?id=123 failed: connection_timeout (retries=3)`

Indexed tokens: `2024`, `01`, `15t10`, `30`, `00z`, `error`, `get`, `api`, `v2`, `users`, `id`, `123`, `failed`, `connection_timeout`, `retries`, `3`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `connection_timeout` | ✅ | ✅ | Token `connection_timeout` in index (underscore kept); exact substring exists |
| `connection timeout` | ❌ | ❌ | Token `connection` not in index (indexed as `connection_timeout`); no exact substring `connection timeout` |
| `timeout` | ❌ | ✅ | Indexed as `connection_timeout` — `timeout` is not the full token. Grep finds substring |
| `api` | ✅ | ✅ | Token `api` in index; exact substring exists |
| `ERROR` | ✅ | ✅ | Lowercased to `error`, matches token; exact substring exists |
| `error` | ✅ | ❌ | Token `error` in index; but log has `ERROR` (case mismatch for grep) |
| `not found` | ❌ | ❌ | `not` is a stop word → only `found` remains, and `found` is not in this log; no substring either |
| `the error` | ✅ | ❌ | `the` removed → `error` matches; no literal `the error` in log |
| `[ERROR]` | ✅ | ✅ | Brackets stripped → token `error`; exact substring exists |
| `id=123` | ✅ | ✅ | Tokens `id`, `123`; exact substring exists |
| `retries=3` | ✅ | ✅ | Tokens `retries`, `3`; exact substring exists |
| `failed: connection_timeout` | ✅ | ✅ | Tokens `failed`, `connection_timeout`; exact substring exists |
| `GET /api` | ✅ | ✅ | Tokens `get`, `api`; exact substring exists |
| `get /api` | ✅ | ❌ | Same tokens (lowercased); but log has `GET` not `get` |
| `10:30:00` | ❌ | ✅ | Search tokens are `10`, `30`, `00`. Log tokens are `15t10`, `30`, `00z`. Token `10` does not match `15t10` and `00` does not match `00z`. Grep finds the exact substring |

### Log line: `error1500 is not a valid response`

Indexed tokens: `error1500`, `valid`, `response`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `1500` | ❌ | ✅ | Token is `error1500`, not `1500`. Grep finds substring |
| `error` | ❌ | ✅ | Token is `error1500`, not `error`. Grep finds substring |
| `is not` | ❌ | ✅ | Both are stop words → removed → empty query. Grep finds exact substring |
| `is not a` | ❌ | ✅ | All stop words → empty. Grep finds exact substring |

### Log line: `user.name=admin connected`

Indexed tokens: `user.name`, `admin`, `connected`

| Search | Term | Grep | Why |
| --- | --- | --- | --- |
| `user` | ❌ | ✅ | Token is `user.name` (dot keeps them together). Grep finds substring |
| `name` | ❌ | ✅ | Token is `user.name`. Grep finds substring |
| `user.name` | ✅ | ✅ | Exact token match; exact substring exists |

## Key differences summary

Term search misses when:

- The search is a **substring of a token** rather than the complete token — for example, `scheduler` does not match the token `com.example.service.scheduler.QuartzJob`.
- The search consists **entirely of stop words** — for example, `is not` produces an empty query after stop word removal.

## When to use which

| Use case | Recommended |
| --- | --- |
| Keyword search (e.g., "find logs with error and timeout") | Term search |
| Exact substring (e.g., "find `status_code=500` exactly") | Grep search |
| Search for part of a dotted name (e.g., `scheduler` within `com.example.scheduler.Job`) | Grep search |
| Search for stop words as part of a phrase (e.g., `is not valid`) | Grep search |
| Fast, broad search across many logs | Term search |