Kubernetes Autoscaling (HPA) with Custom Metrics

You must have the following to scale components based on custom metrics:

• To collect metrics from applications and store them in the Kloudfuse database.
• To extend the Kubernetes Custom Metrics API with metrics supplied by a collector, the k8s-prometheus-adapter. The adapter is an implementation of the custom metrics API that supports arbitrary metrics.

Consider a case where Kloudfuse ingests an application-specific metric, some_metric. You plan to modify it, and use its value for autoscaling the Kubernetes pods. Follow these steps:

1. Install Prometheus adapter with proper configuration to emit a custom_metric based on a query for some_metric.

2. Create prometheus-values.yaml file for the Prometheus Adapter, configuring the proper endpoint, authentication, and custom metric rules.

    metricsRelistInterval: 5m

prometheus:
         # Replace with your endpoint for Kloudfuse
         url: <<https://your.hostname.kloudfuse>>
         port: 443
         path: ""

# Rules to generate `some_custom_metric` as an external metric
    rules:
         default: false
         external:
      - seriesQuery: '{__name__=~"some_metric"}'
        resources:
          template: <<.Resource>>
        name:
          as: "some_custom_metric"
        metricsQuery: avg (some_metric)

# Base64 Encoded Auth string in the format <<username:password>>
    extraArguments:
      - --prometheus-header=Authorization=Basic YWRtaW46cGFzc3dvcmQ=
    ```

3. Deploy Prometheus adapter on your cluster.

```console
    helm install --name prometheus-adapter prometheus-community/prometheus-adapter -f prometheus-values.yaml
    ```

4. Verify that the adapter emits `custom-metric` correctly, using the Kubernetes External Metrics API.

See [Prometheus Adapter](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-adapter) code and documentation for additional configuration information.

```yaml
    kubectl get --raw /apis/external.metrics.k8s.io/v1beta1 | jq .

#Should show some output like this
    {
         "kind": "APIResourceList",
         "apiVersion": "v1",
         "groupVersion": "external.metrics.k8s.io/v1beta1",
         "resources": [\
           {\
             "name": "some_custom_metric",\
             "singularName": "",\
             "namespaced": true,\
             "kind": "ExternalMetricValueList",\
             "verbs": [\
               "get"\
             ]\
           }\
         ]
    }
    ```

5. After `custom-metric` becomes available to use within the Kubernetes Metrics API, configure HPA to use the metric.

```yaml
    apiVersion: autoscaling/v2
    kind: HorizontalPodAutoscaler
    metadata:
         name: <<some_name>>
    spec:
         scaleTargetRef:
           apiVersion: apps/v1
           kind: Deployment
           name: <<some_name>>
         minReplicas: 1
         maxReplicas: 10
         metrics:
      - type: External
        external:
          metric:
            name: some_custom_metric
          target:
            type: AverageValue
            averageValue: 100
    ```

6. Explore how to configure HPA with Custom metrics in Kubernetes documentation for [Autoscaling on multiple metrics and custom metrics](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-multiple-metrics-and-custom-metrics).