# FuseQL Best Practices

## Table of Contents

- [Know the three layers of a log line](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#data-layers)
- [Start with the most selective stream selector](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#selective-selectors)
- [Filter on ingest-time facets, not on strings](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#facets-not-strings)
- [Use term search for log body searches](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#term-search)
- [Reach for a parser only when the facet does not exist](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#skip-parser)
- [Keep metric queries pushdown-friendly](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#metric-pushdown)
- [Use `compose` carefully](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#compose-carefully)
- [Bound the scope and the time range](https://docs.kloudfuse.com/platform/latest/query-languages/fuseql/best-practices/#scope-and-time)

The fastest FuseQL queries on Kloudfuse touch the least data. Every log line has three layers of structure — indexed stream labels, ingest-time facets, and the raw log body — and the right operator for each layer is different. This page explains the layers, how the query engine evaluates them, and the habits that keep queries fast at scale.

## Know the three layers of a log line

| Layer | Examples | How to filter | Cost |
| --- | --- | --- | --- |
| Indexed stream labels | `source`, `level`, `kube_namespace`, `kube_service`, `pod_name`, `cluster_name`, `host`, `availability_zone`, `project`, `docker_image`, `kube_container_name`, `instance_id`, `instance_type` | Stream selector or pre-pipe condition: `source="nginx" level="error"` | Cheapest — resolved against the label index before any data is read |
| Ingest-time facets | Fields Kloudfuse extracted when the log arrived: `statusCode`, `endpoint`, `duration`, `userId`, and any other key your pipeline parses | Post-pipe or pre-pipe facet filter: `@statusCode=200` or `| statusCode="200"` | Cheap — resolved from stored facet columns; no log-body scan |
| Raw log body | The original message text | Term search: `'error'`; grep: `"err*"`; regex: `=~"pattern"` | Most expensive — requires scanning the log_line column; term search is index-backed and much faster than grep or regex |

Structure every query in that order: select streams by label, narrow by facet, and only then search the remaining text.

## Start with the most selective stream selector

The stream selector is evaluated first and determines how much data the rest of the pipeline reads. Name the source and add every indexed label you know:

```none
source="grafana" level="error"
```

The full set of indexed labels — resolved as direct column lookups before any row is fetched — is:

`source`, `level`, `kube_namespace`, `kube_service`, `pod_name`, `cluster_name`, `host`, `availability_zone`, `project`, `docker_image`, `kube_container_name`, `instance_id`, `instance_type`

Two habits keep selectors fast:

- Prefer `=` to `=~`. An equality matcher is a direct columnar lookup; a regex matcher runs RE2 against every label value in the result set.
- Never search for what a label already states. `source="app" |= "ERROR"` scans every line; `source="app" level="error"` reads only the error streams and catches all capitalizations because Kloudfuse normalises `level` at ingest.

Avoid match-everything patterns such as `source=~".+"` — they make the stream selector a no-op and turn the entire time range into the query’s working set.

## Filter on ingest-time facets, not on strings

Kloudfuse extracts facets from structured content at ingest and stores them alongside the line. Filter on them directly — no parser stage is needed:

```none
source="grafana" | statusCode="200"
```

Numeric, duration, and bytes comparisons work the same way:

```none
source="grafana" | duration > 100ms
```

The pre-pipe facet syntax with `@` is equivalent and slightly more concise:

```none
source="grafana" @statusCode=200
```

Compare this with the string-search version, `source="grafana" |= "statusCode=200"`, which scans every line, breaks if the field order or spacing changes, and also matches lines that merely mention that text.

Remember: facets that are in the stream selector position only match indexed stream labels. Write facet conditions after the opening selector or with the `@` prefix:

```none
source="grafana" @statusCode=503 @endpoint="/api/query"
```

## Use term search for log body searches

When you need to search the raw log body, prefer term search over grep or regex. Term search resolves through the Lucene inverted index that Kloudfuse builds from every log line at ingest — it is an index lookup, not a scan:

```none
'OutOfMemoryError'
```

Term search is case-insensitive by design and matches whole tokens only. Combine with stream and facet conditions so the index result set is already narrowed:

```none
source="payment" level="error" @endpoint="checkout" 'OutOfMemoryError'
```

Grep (`"pattern"`) supports wildcards and is a good choice when you need partial-word matching or a prefix:

```none
"conn*"         # matches connection, connect, connected
"*refused"      # matches suffix
```

Internally, grep with a literal prefix pre-filters candidates through the index before doing per-row work — significantly faster than a pure regex scan. Reserve `=~` and `!~` for cases where neither term search nor grep expresses the intent.

## Reach for a parser only when the facet does not exist

Parsers (`json`, `logfmt`, `pattern`, `regexp`) re-extract fields from the raw log body on every line, on every query. That work is wasted when the field is already an ingest-time facet. Use the facet directly:

```none
source="nginx" @statusCode=503
```

Use a parser when a field is genuinely not extracted at ingest — a one-off pattern inside the message body, or a format your pipeline does not parse:

```none
source="nginx" | regexp `(?P<method>[A-Z]+) (?P<path>[^ ]+)` | method="POST"
```

When a parser is necessary, note that basic `json`, `logfmt`, `pattern`, and `regexp` parsers without parameters are pushed down to the storage layer. Parsers with expression parameters are evaluated in the query engine, which is slower. Keep parser stages as simple as possible.

## Keep metric queries pushdown-friendly

FuseQL pushes metric computation down into the Pinot storage layer when the query shape allows it. Pushed-down queries are dramatically faster than those computed in the engine. The following stages are pushdown-friendly:

- Stream label filters and indexed facet filters
- Line filters (`|=`, `|~`, `!=`, `!~`)
- Basic parsers (`json`, `logfmt`, `pattern`, `regexp`) without parameters
- `label_format` used only to **rename** a label (`| label_format new=old`)
- Metric aggregations: `count`, `sum`, `avg`, `min`, `max`, and the percentile operators (`p50`, `p95`, `p99`, …) when combined with a `timeslice` grouping

The following stages disable pushdown for the remainder of the pipeline:

- `line_format` with template syntax (`{{ .field }}`)
- `label_format` used to add or modify labels (not just rename)
- Any parser invoked with expression parameters

```none
# Pushdown-eligible — timeslice + count pushed to storage
source="grafana" | timeslice 5m | count by (_timeslice, level)

# Not pushdown-eligible — line_format forces engine evaluation
source="grafana" | line_format "{{.msg}}" | timeslice 5m | count by (_timeslice)
```

If a `line_format` stage is needed for display, keep it in a separate log-retrieval query and not in a metric query.

## Use `compose` carefully

The `compose` operator runs a subquery and uses its results as a filter. It has a hard result limit of 10,000 entries (default: 2,500). Exceeding the limit truncates the subquery result silently and may cause the outer query to miss matching rows:

```none
# Safe — subquery returns a small, bounded set
source="payment" | error_code exists
  | compose error_code maxresults=500
```

Keep compose subqueries as selective as possible. If you need to match a large set, consider writing the condition directly as a facet filter instead of using compose.

## Bound the scope and the time range

Query cost scales with the data volume selected. The time range is a multiplier on everything above. While iterating on a query, work against a short window — the last 5 or 15 minutes — and widen it only when the query is final.

For dashboards, size the range window `[…​]` to the panel’s step rather than defaulting to large windows: `rate({…​}[1m])` at a 1-minute step reads each log line once. A 1-hour window at the same step reads 60× more data for no extra granularity.

Add `limit N` to log-retrieval queries to cap the result set. The maximum is 10,000 rows; queries that omit a limit use the default and may return more data than needed.