How We Built FuseQL: A Query Language for Log Analytics at Scale

Why Build a New Query Language Instead of Extending LogQL?

We started with LogQL. Kloudfuse originally used Grafana's LogQL as the primary log query language. Three limitations pushed us toward building FuseQL:

Limited expressiveness. Most log query languages are designed around single-aggregation, single-vector output models. They let you count errors or compute a rate, but not return multiple independent aggregations as columns in a single tabular result, the way an engineer naturally thinks about analysis: show me sum, avg, and max side-by-side, grouped by service. FuseQL supports this natively. For teams used to writing multi-aggregation queries in tools like Splunk or Sumo Logic, this was the difference between adopting Kloudfuse for logs and sticking with their existing stack.

Rabiya, Customer Success Architect at Kloudfuse, noted: "The most common feedback from teams migrating from Splunk or Sumo Logic was that LogQL felt like a step backward. FuseQL changed that conversation entirely. Engineers could write the same multi-aggregation queries they were used to, and the pipe syntax meant the learning curve was days, not weeks."

Performance at scale. LogQL queries slow down significantly with large datasets and high-cardinality log attributes. In environments generating millions of log events per second, this creates a ceiling on what you can investigate interactively.

Licensing constraints. LogQL is governed by the AGPL license. Any modifications must be shared with the community, which may not align with all customers' deployment requirements. Building on AGPL code limits how you can distribute and customize the query engine.

As JT, Staff Engineer at Kloudfuse, put it: "Building a new query language is inherently complex. Our process paralleled compiler design, requiring critical architectural decisions around language grammar and execution strategies." The alternative, continuing to patch around LogQL's limitations, would have created more complexity over time, not less.

How Does FuseQL Syntax Work?

FuseQL uses a pipe-based syntax where operations chain with |. If you've written Splunk SPL, the pattern will feel familiar:

\| \| ...

The result of each operator feeds into the next. Every FuseQL query produces a table following a schema defined by column headers. Here are examples that demonstrate the progression from simple to complex:

Count all logs in 5-second buckets:

* | timeslice 5s | count by (_timeslice)

Count logs grouped by severity level:

* | timeslice 30s | count by (_timeslice, level)

Average duration facet over time:

* | timeslice 5s | avg(@duration:duration_seconds) by (_timeslice)

Detect outliers in error counts using DBSCAN:

level="error" | timeslice 120s | count by (_timeslice, kube_namespace) | outlier (_count) by 120s, model=dbscan, eps=3

That last example is where FuseQL's design philosophy shows. An engineer investigating an error spike doesn't want to export data to a notebook for anomaly detection. They want to identify the outlier namespace directly in the query, using the same tool they use for everything else.

What Operators Does FuseQL Support?

FuseQL ships with over 60 operators across several categories. This is not a comprehensive list, but it covers the categories that matter most for understanding the language's scope:

Aggregation:avg, count, count_unique, first, last, max, min, percentiles, stddev, sum. Supports multiple aggregations in a single query (a limitation that LogQL does not overcome).
Algorithmic/ML:anomalies overlays expected behavior bands on time series. outliers highlights outlier series using DBSCAN clustering. forecast predicts future values from historical data.
Parse: Variable pattern extraction with regex, anchor-based parsing, native JSON array parsing, and split operations.
Subqueries (new in 4.0): Nested analysis where results from one query feed into another.
Compare:compare timeshift analyzes data across different time periods for before-and-after analysis during deployments or incidents.
DIFF: Compares two time ranges or result sets to identify additions, deletions, and modifications.
Lookup: Enriches log data at query time from external CSV-based lookup tables without re-ingesting data.
Search: Boolean operators, regex matching, facet filtering.
Window:accum (running accumulation), rollingstd (rolling standard deviation), smooth (moving average), total (running total).
Miscellaneous: Over 40 utility functions including base64Decode, hexToDec, ipv4ToNumber, isPrivateIP, luhn (credit card validation).

How Does the Query Engine Architecture Work?

FuseQL queries run against Apache Pinot, the distributed real-time OLAP datastore at the core of Kloudfuse's storage layer. The architecture has several properties that matter for query performance:

Schema-on-read. Logs are ingested without requiring a predefined schema. The query engine interprets structure at read time.

Columnar storage with fingerprinting. Kloudfuse's patent-pending log fingerprinting technology separates each log line into static components and dynamic values.

Computation pushed to storage. FuseQL operators like matches and in execute at the Pinot storage layer rather than in a separate query processing layer.

Dual representation. FuseQL queries can produce both time series results and streaming raw log results.

Ashvin Kumaran noted: "Enhancing the efficiency of these operations has significantly improved our system's performance."

FuseQL at a glance


Capability	What it means for your investigation
Pipe-based syntax	Chain filter → aggregate → filter → visualize in a single query.
Multi-column aggregation	Return sum, avg, max, percentiles as separate columns in one tabular result.
Built-in ML operators	Anomaly detection, outlier detection, and forecasting run inside the query.
Subqueries (4.0)	Nest queries: find hosts with highest error rate, then pull their detailed logs.
60+ operators	Aggregation, parsing, windowing, comparison, arithmetic, trigonometry, lookup enrichment, and 40+ utility functions.
Schema-on-read	New log formats are immediately queryable.
Storage-layer pushdown	Operators execute at the Pinot storage layer, not in memory.
LogQL backward compatibility	Existing LogQL queries continue to work.

FuseQL is purpose-built for observability log analytics. It doesn't try to be a general-purpose data language.

What's the Relationship Between FuseQL and PromQL?

They're complementary, not competing. PromQL is the industry standard for metrics queries, and Kloudfuse supports it as a first-class query language. FuseQL handles log analytics where PromQL's time-series-oriented model doesn't apply.

The Design Decision: Compiler Design for a Query Language

Building FuseQL required the same rigor as building a compiler: defining a formal grammar, implementing a parser, building an optimizer, and designing an execution engine.

Next Steps

The FuseQL documentation covers the full operator reference with examples. The FuseQL additional examples page provides real-world query patterns for common investigation scenarios.

What does your team's log investigation workflow look like? Are you writing queries from scratch every time, or have you built up a library of saved patterns? We're always interested in how teams bridge the gap between "something is wrong" and "here's the root cause."