Algorithmic operators

FuseQL provides algorithmic operators for detecting unexpected behavior in time-series data. These operators apply statistical models to identify anomalies and highlight outlier series.

Anomaly detection

Anomaly detection is a powerful monitoring feature that uses algorithmic analysis to automatically identify unexpected behavior in metric data. Traditional threshold-based alerting often fails to account for trends, seasonality, or complex fluctuations in metrics.

Anomaly detection algorithms overcome this limitation by analyzing historical patterns to establish dynamic boundaries (bounds), making it possible to detect deviations from normal behavior even as the data changes over time.

In practice, anomaly functions overlay a band on the metric, showing the expected behavior of a series based on past values.

Kloudfuse provides four anomaly detection models.

Models

• Basic anomaly detection
  Implements the Rolling quantile algorithm.

• Agile anomaly detection
  Uses the SARIMA model.

• Robust anomaly detection
  The robust anomaly detection algorithm uses a Seasonal Decompose.

• Agile-Robust anomaly detection
  Applies the Prophet model.

Basic anomaly detection

This algorithm calculates a predicted range using the 25th and 75th quantiles and the interquartile range (IQR) within a rolling window. This range determines the expected normal behavior; deviations outside this range are anomalies.

Basic anomaly detection is ideal for monitoring metrics with frequent, non-seasonal fluctuations, where rapid response to changes is essential. Use it to detect unexpected spikes or drops without needing to account for cyclic patterns or trends.

Parameters

• window
  Defines the size of the rolling window for quantile computation. A larger window smooths the data, but reduces sensitivity to sudden changes.

• bounds
  Set the sensitivity of anomaly detection. Narrower bounds make the algorithm more sensitive to deviations, while wider bounds classify more data as normal.

• Bounds:1 (Narrow) Offers high sensitivity by setting a tighter range around predicted values, detecting even minor deviations.

  • Bound 2 (Moderate): Provides a moderate range, making the algorithm less sensitive to minor fluctuations and ideal for monitoring with fewer false positives.
  • Bound 3 (Wide): Defines the widest range, capturing only significant deviations.

Example Query

* | timeslice 1200s | count_unique(@error) by (_timeslice) | anomaly (_count_unique) by 1200s, model=basic, bounds=1, window=2h, band=3

Agile anomaly detection

Parameters

• Seasonality
  The algorithm auto-detects the seasonality, either hourly or daily:
  • Hourly: Logs metrics that display hourly cyclic behavior.
  • Daily: Captures daily seasonality.
• bounds
  Same as Basic anomaly detection.

Example Query

* | timeslice 1200s | count by (_timeslice) | anomaly (_count) by 1200s, model=agile, bounds=1, band=3

Robust anomaly detection

The Robust anomaly detection algorithm uses a Seasonal Decompose technique to identify anomalies in time series data.

Parameters

• Rolling Window size
  Calculate the standard deviation (stddev) for anomaly detection.
• bounds
  Same as Basic anomaly detection.

Example Query

* | timeslice 1800s | count by (_timeslice) | anomaly (_count) by 1800s, model=robust, seasonality=daily, bounds=1, trend=additive, window=30m, band=3

Agile-Robust anomaly detection

Applies the Prophet model to detect anomalies in log metrics with recurring patterns and occasional level shifts.

Example Query with Bound 1

* | timeslice 120s | last(@durationHourly:number) by (_timeslice) | anomaly (_last) by 120s, model=agileRobust, seasonality=hourly, bounds=1, band=3

Example Query with Bound 3

* | timeslice 120s | last(@durationHourly:number) by (_timeslice) | anomaly (_last) by 120s, model=agileRobust, seasonality=hourly, bounds=3, band=3

Outlier detection

Kloudfuse uses the Outliers function to highlight outlier time series.

DBSCAN

Kloudfuse provides the DBSCAN implementation of outlier detection.

Parameters

• tolerance
  Determines the clustering radius of the neighborhood around each point. A lower tolerance detects more subtle outliers, while a higher tolerance detects only the most significant deviations.

Example with Tolerance of 0.8

* | timeslice 60s | count by (_timeslice, @sourceIPAddress) | cbrt(_count) as _cbrt | outlier (_cbrt) by 60s, model=dbscan, eps=0.8

Example with Tolerance of 5

* | timeslice 60s | count by (_timeslice, @sourceIPAddress) | cbrt(_count) as _cbrt | outlier (_cbrt) by 60s, model=dbscan, eps=5