Anomaly detection

Anomaly detection is a powerful monitoring feature that uses algorithmic analysis to automatically identify unexpected behavior in metric data. Traditional threshold-based alerting often fails to account for trends, seasonality, or complex fluctuations in metrics.

Anomaly detection algorithms overcome this limitation by analyzing historical patterns to establish dynamic boundaries (bounds), making it possible to detect deviations from normal behavior even as the data changes over time.

In practice, anomaly functions overlay a band on the metric, showing the expected behavior of a series based on past values.

Kloudfuse provides these possible implementations of anomaly detection:

• Basic: Implements the Rolling quantile algorithm.
  See Basic anomaly detection.
• Agile: Uses the Seasonal AutoRegressive Integrated Moving Average (SARIMA) model.
  See Agile anomaly detection.
• Robust: The robust anomaly detection algorithm uses a seasonal decomposition.
  See Robust anomaly detection.
• Agile-Robust: Applies the Prophet model.
  See Agile-Robust anomaly detection.

Basic anomaly detection

Implements the Rolling quantile algorithm.

This algorithm calculates a predicted range using the 25th and 75th quantiles and the interquartile range (IQR) within a rolling window. This range determines the expected normal behavior; deviations outside this range are anomalies.

Basic anomaly detection is ideal for monitoring metrics with frequent, non-seasonal fluctuations, where rapid response to changes is essential. Use it to detect unexpected spikes or drops without needing to account for cyclic patterns or trends.

Parameters

• window: Defines the size of the rolling window for quantile computation. A larger window smooths the data, but reduces sensitivity to sudden changes.
• bounds: Set the sensitivity of anomaly detection. Narrower bounds make the algorithm more sensitive to deviations, while wider bounds classify more data as normal.
  • Bounds: 1 (Narrow): Offers high sensitivity by setting a tighter range around predicted values, detecting even minor deviations.
  • Bound 2 (Moderate): Provides a moderate range, making the algorithm less sensitive to minor fluctuations and ideal for monitoring with fewer false positives.
  • Bound 3 (Wide): Defines the widest range, capturing only significant deviations.

Example

* | timeslice 1200s | count_unique(@error) by (_timeslice) | anomaly (_count_unique) by 1200s, model=basic, bounds=1, window=2h, band=3

The time series graph displays a unique count of errors over a period. The gray band represents the expected range based on recent data, while red markers indicate anomalies — data points outside the predicted range. Here, a sudden increase in errors during peak hours is flagged as an anomaly, helping with quick detection and investigation.

Agile anomaly detection

Parameters

• Seasonality: The algorithm auto-detects the seasonality, either hourly or daily:
  • Hourly: Logs metrics that display hourly cyclic behavior.
  • Daily: Captures daily seasonality, suitable for log metrics with a daily recurring pattern.
• bounds: Set the sensitivity of anomaly detection. Narrower bounds make the algorithm more sensitive to deviations, while wider bounds classify more data as normal.
  • Bounds: 1 (Narrow): Offers high sensitivity by setting a tighter range around predicted values.
  • Bound 2 (Moderate): Provides a moderate range, making the algorithm less sensitive to minor fluctuations.
  • Bound 3 (Wide): Defines the widest range, capturing only significant deviations.

Example

* | timeslice 1200s | count by (_timeslice) | anomaly (_count) by 1200s, model=agile, bounds=1, band=3

Robust anomaly detection

The Robust anomaly detection algorithm uses a seasonal decomposition technique to identify anomalies in time series data.

Parameters

• Rolling Window size: Calculate the standard deviation (stddev) for anomaly detection, and to set the bounds for the expected values.
• bounds: Set the sensitivity of anomaly detection. Narrower bounds make the algorithm more sensitive to deviations, while wider bounds classify more data as normal.
  • Bounds: 1 (Narrow): Offers high sensitivity by setting a tighter range around predicted values.
  • Bound 2 (Moderate): Provides a moderate range, making the algorithm less sensitive to minor fluctuations.
  • Bound 3 (Wide): Defines the widest range, capturing only significant deviations.

Example

* | timeslice 1800s | count by (_timeslice) | anomaly (_count) by 1800s, model=robust, seasonality=daily, bounds=1, trend=additive, window=30m, band=3

Agile-Robust anomaly detection

Applies the Prophet model to detect anomalies in log metrics with recurring patterns and occasional level shifts.

Parameters

• Seasonality: The algorithm auto-detects the seasonality, either hourly or daily:
  • Hourly: Captures hourly cyclic behavior.
  • Daily: Suitable for log metrics with a daily recurring pattern.

Example with Bound 1

* | timeslice 120s | last(@durationHourly:number) by (_timeslice) | anomaly (_last) by 120s, model=agileRobust, seasonality=hourly, bounds=1, band=3

Example with Bound 3

* | timeslice 120s | last(@durationHourly:number) by (_timeslice) | anomaly (_last) by 120s, model=agileRobust, seasonality=hourly, bounds=3, band=3