AWS CloudWatch Metric Aggregations

Table of Contents

• Overview
• How the Plain Metric Name Is Chosen
• Metrics by Namespace
  • AWS/AmazonMQ
  • AWS/ApiGateway
  • AWS/ApplicationELB
  • AWS/Athena
  • AWS/AutoScaling
  • AWS/CloudFront
  • AWS/DynamoDB
  • AWS/EBS
  • AWS/EC2
  • AWS/ECS
  • AWS/EFS
  • AWS/ELB
  • AWS/ES
  • AWS/Firehose
  • AWS/FSx
  • AWS/Glue
  • AWS/Kafka
  • AWS/Kinesis
  • AWS/Lambda
  • AWS/NATGateway
  • AWS/NetworkELB
  • AWS/RDS
  • AWS/Redshift
  • AWS/Route53
  • AWS/S3
  • AWS/SNS
  • AWS/SQS
  • ECS/ContainerInsights
• Extending the Defaults
  • Keeping Additional Statistics
  • Changing the Main Statistic

AWS CloudWatch delivers each metric as a bundle of statistics in a single record: sum, count, minimum, and maximum, with average derived as sum / count. Rather than keep every statistic for every metric, Kloudfuse takes an opinionated view of which one statistic best represents each metric. This page lists those selections.

Overview

For every metric that Kloudfuse special-cases:

• One statistic is chosen as the main metric and is stored under the plain, suffix-less name.

• Any other statistics worth keeping are stored as separate series, with the aggregation appended as a suffix (for example, _maximum).

• A metric that Kloudfuse does not special-case keeps only its CloudWatch sum, stored under the plain name.

This means the plain metric name and a suffixed variant are not two views of the same number. The plain name is whichever statistic is marked main. For example, aws.ec2.cpuutilization is the average CPU utilization: because average is the main statistic, there is no separate _average series, and the maximum is published separately as aws.ec2.cpuutilization_maximum.

CloudWatch metrics reach Kloudfuse through the Metrics Stream and Kinesis Firehose path described in AWS CloudWatch Metrics Integration.

How the Plain Metric Name Is Chosen

Kloudfuse stores the main statistic under the plain metric name and appends a suffix to every additional statistic it keeps. The suffix names are:

The raw CloudWatch metric name is normalized into the Kloudfuse metric name shown in the tables below:

• If Kloudfuse defines an explicit override for the metric, that name is used as-is. For example, AllocatedStorage under AWS/RDS becomes total_storage_space.

• Otherwise the name is converted to snake case as follows: a trailing Count is dropped; -, ., ``, and % are removed; and an underscore is inserted between a lowercase letter and a following uppercase letter or digit. Consecutive capitals (acronyms) stay together, so CPUUtilization becomes cpuutilization while DatabaseConnections becomes database_connections.

The full metric name also carries the normalized namespace as a prefix, following the aws.<namespace>.<name> convention. For example, CPUUtilization under AWS/EC2 becomes aws.ec2.cpuutilization. See AWS CloudWatch Metrics Integration.

Metrics by Namespace

In the Metric column, the CloudWatch metric name is shown on top, with the Kloudfuse metric name in parentheses beneath it. The parenthesized name is what appears in the Metrics Explorer in the Kloudfuse UI, prefixed with the namespace — for example, CPUUtilization appears as aws.ec2.cpuutilization. The remaining columns show the statistic stored under the plain (suffix-less) name and any additional suffixed series that are also emitted. Jump to a service from the table of contents, or use your browser’s find (Ctrl+F / Cmd+F) to locate a metric.

sum (default) means Kloudfuse does not mark a main statistic, so the plain metric is the CloudWatch default sum.

AWS/AmazonMQ

AWS/ApiGateway

AWS/ApplicationELB

AWS/Athena

AWS/AutoScaling

AWS/CloudFront

AWS/DynamoDB

AWS/EBS

AWS/EC2

AWS/ECS

AWS/EFS

AWS/ELB

AWS/ES

AWS/Firehose

AWS/FSx

AWS/Glue

AWS/Kafka

AWS/Kinesis

AWS/Lambda

AWS/NATGateway

AWS/NetworkELB

AWS/RDS

AWS/Redshift

AWS/Route53

AWS/S3

AWS/SNS

AWS/SQS

ECS/ContainerInsights

Extending the Defaults

The selections on this page are Kloudfuse defaults. Through the additionalCloudWatchMetricAggregations setting in the Helm configuration, a deployment can keep additional statistics for a metric, or change which statistic is stored under the plain metric name.

Add the setting under ingester.config.aggregator.metric in your custom-values.yaml file. Each list entry configures one statistic for one metric, with these fields:

namespace

The raw CloudWatch namespace, such as AWS/EC2 or AWS/EBS.

name

The raw CloudWatch metric name, such as CPUUtilization. Use the CloudWatch name (the top name in each row of the tables above), not the normalized Kloudfuse name.

aggregation

The CloudWatch statistic to configure, written as its short name: sum, average, min, max, or count, plus percentiles such as p99 when the metric stream carries them. Write min and max, not minimum or maximum, and count, not samplecount; these are renamed only in the emitted series name (see How the Plain Metric Name Is Chosen), not in this field. Use one of these short tokens exactly; do not use CloudWatch’s own capitalized statistic names such as Average or SampleCount. An unrecognized value, or an entry with no aggregation at all, is silently ignored.

isMain

Optional; defaults to false. When false, the statistic is kept as an additional, suffixed series. When true, the statistic becomes the main metric, stored under the plain, suffix-less name, replacing the previous main. See Changing the Main Statistic.

Keeping Additional Statistics

Keeping an additional statistic is most useful for a metric that Kloudfuse does not special-case. Such a metric keeps only its CloudWatch sum, stored under the plain name (see the TIP in Overview). Adding an aggregation for it makes Kloudfuse also keep that statistic as a separate, suffixed series, so a custom metric can carry, for example, its average and max in addition to the default sum.

Leave isMain unset (or false) for these entries:

Keep additional statistics

ingester:
  config:
    aggregator:
      metric:
        # additionalCloudWatchMetricAggregations - Keep extra CloudWatch
        # statistics for a metric, in addition to the Kloudfuse defaults.
        additionalCloudWatchMetricAggregations:
          - namespace: AWS/EC2
            name: CPUUtilization
            aggregation: average
          - namespace: AWS/EC2
            name: CPUUtilization
            aggregation: max
          - namespace: AWS/EBS
            name: VolumeReadBytes
            aggregation: average

yamlCopied!

Each statistic you add is stored as a separate series, with the aggregation appended as a suffix following the rules in How the Plain Metric Name Is Chosen: average becomes _average, max becomes _maximum, min becomes _minimum, and count becomes _samplecount. The plain, suffix-less metric name is unchanged.

Entries can also be written in compact, inline form, which keeps the list readable when you are configuring many metrics across several namespaces:

Compact inline form

ingester:
  config:
    aggregator:
      metric:
        additionalCloudWatchMetricAggregations:
          - {namespace: AWS/EC2, name: CPUUtilization, aggregation: max}
          - {namespace: AWS/SQS, name: ApproximateAgeOfOldestMessage, aggregation: average}
          - {namespace: AWS/ELB, name: Latency, aggregation: p99}
          - {namespace: AWS/RDS, name: ReadLatency, aggregation: max}
          - {namespace: AWS/Lambda, name: Duration, aggregation: min}
          - {namespace: AWS/DynamoDB, name: ConsumedReadCapacityUnits, aggregation: sum}

yamlCopied!

Changing the Main Statistic

By default, the statistic stored under the plain, suffix-less name is whichever one the tables above mark as main; for a metric that Kloudfuse does not special-case, that default is the CloudWatch sum. Set isMain: true on an aggregation to store that statistic under the plain name instead. How many entries you need depends on whether Kloudfuse already special-cases the metric.

For a metric that Kloudfuse does not special-case (it defaults to sum only), a single entry with isMain: true is enough:

Promote a statistic to the main metric (metric with no default)

ingester:
  config:
    aggregator:
      metric:
        additionalCloudWatchMetricAggregations:
          - namespace: My/CustomNamespace
            name: MyCustomMetric
            aggregation: average
            isMain: true

yamlCopied!

For a metric that Kloudfuse already special-cases (one whose main is already a chosen statistic in the tables above), supply two entries: one to demote the current main with isMain: false, and one to promote the new main with isMain: true. For example, to make max the main statistic for AWS/EC2``CPUUtilization (whose default main is average):

Change the main statistic (already special-cased metric)

ingester:
  config:
    aggregator:
      metric:
        additionalCloudWatchMetricAggregations:
          # Demote the current main; it is then emitted as _average.
          - namespace: AWS/EC2
            name: CPUUtilization
            aggregation: average
            isMain: false
          # Promote max; it is then stored under the plain name.
          - namespace: AWS/EC2
            name: CPUUtilization
            aggregation: max
            isMain: true

yamlCopied!

After editing custom-values.yaml, apply the change with a Helm upgrade:

helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
  -n kfuse \
  --version <VERSION> \ (1)
  -f custom-values.yaml

Copied!

See AWS CloudWatch Metrics Integration for CloudWatch ingestion setup, and Relabel Rules for other ingestion-time metric configuration.

Was this page helpful?

Yes

No

What did you like?

Accurate

Accurately describes the product or feature

Solved my problem

Helped me resolve an issue

Easy to understand

Easy to follow and comprehend

Helped me decide to use the product

Convinced me to adopt a product or feature

Another reason

Cancel

Submit

What went wrong?

Inaccurate

Doesn't accurately describe this product or feature

Couldn't find what I was looking for

Missing important information

Hard to understand

Too complicated or unclear

Code sample errors

One or more code samples are incorrect

Another reason

Cancel

Submit

Thanks for sharing your thoughts! Your feedback helps us improve

✕