Ingestion Authentication with API Key

To enable authentication for ingestion using API keys, perform these steps:

• Generate the AUTH_TOKEN

• Configure Telemetry Agents and Sources

• Configure kfuse

• Restart the Ingester

Generate the AUTH_TOKEN

1. Generate an AUTH_TOKEN, and store its value in a safe location. You use it repeatedly.

   AUTH_TOKEN=`cat /dev/urandom | env LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1`

2. Encode the AUTH_TOKEN using Base64; this generates the AUTH_TOKEN_ENCODED.

   AUTH_TOKEN_ENCODED=`echo -n $AUTH_TOKEN | base64`

Configure Telemetry Agents and Sources

Prometheus Remote Write

1. Update the prometheus remote write configuration:

   prometheus.yml:
        remote_write:
     - url: https://<customer>.kloudfuse.io/ingester/write
       authorization:
         credentials: <AUTH_TOKEN>
   

2. When using the Prometheus operator, use the following configuration that contains the AuthToken:

   remoteWrite:
     - authorization:
         credentials:
           key: <AUTH_TOKEN> (1)
           name: kf-auth-ingest (2)
       url: https://<customer>.kloudfuse.io/ingester/write
   

Fluent Bit

1. In the HTTP plugin section of the fluent-bit configuration file, update or add the header field using the new AUTH_TOKEN value (from the previous step):

   [OUTPUT]
          Name http
          Match <match_pattern>
          Host <kfuse_ingress_ip>
          Port 443
          TLS on
          URI /ingester/v1/fluent_bit
          header Kf-Api-Key <AUTH_TOKEN>
   

2. Use the Kubernetes Secret to set the Kf-Api-Key.

3. Create the secret:

   apiVersion: v1
   kind: Secret
   metadata:
     name: <<secret-name>>
   type: Opaque
   data:
     AUTH_KEY_ENV: AUTH_TOKEN_ENCODED
   

4. Update the fluent-bit helm using the custom-values.yaml file:

   env:
   - name: AUTH_KEY_ENV
   valueFrom:
      secretKeyRef:
        name: <<secret-name>>
        key: AUTH_KEY_ENV
   

config: outputs: | [OUTPUT] Name http Header Kf-Api-Key ${AUTH_KEY_ENV} Match * Host dev.kloudfuse.io Port 443 TLS on URI /ingester/v1/fluent_bit Format {json|json_lines|json_stream|msgpack} ```

Fluentd

1. In the HTTP plugin section of the Fluentd configuration file, update or add the headers field using the new Kf_Api-Key and AUTH_TOKEN value (from the previous step):

   <match *> # Match everything
   @type http
   endpoint http://<KFUSE_INGESTER_IP>:80/ingester/v1/fluentd
   headers {"Kf-Api-Key" : "<AUTH_TOKEN>"}
   ...
   </match>
   

Filebeat

In the output section, include the api_key field to update or add a filebeat configuration.

output.elasticsearch:
hosts: ["http://<ingress-ip>:80/ingester/api/v1/filebeat"]
api_key: "<AUTH_TOKEN>"

OLTP Collector for Metrics, Logs, and Traces

In the exporters section, include the api_key field using AUTH_TOKEN value to update or add an OLTP configuration.

exporters:
otlphttp:
    endpoint: https://<ingress-address>/ingester/otlp/metrics
    traces_endpoint: https://<ingress-address>/ingester/otlp/traces
    headers:
      kf-api-key: <AUTH_TOKEN>

DataDog agent

Include the api_key field using AUTH_TOKEN value to update or add a DataDog configuration.

datadog:
apiKey: <AUTH_TOKEN>

AWS CloudWatch Metrics and Logs (Kinesis)

When configuring kinesis firehose data stream to send logs or metrics from Cloudwatch, use the AUTH_TOKEN value as the access token.

If the firehose data stream is already operational, update it to use the AUTH_TOKEN value as an access token.

AWS Eventbridge Events

Configure kfuse

Use the base64-encoded value of the AUTH_TOKEN, AUTH_TOKEN_ENCODED, to create a kubernetes secret kfuse-auth-ingest:

apiVersion: v1
kind: Secret
metadata:
name: kfuse-auth-ingest
type: Opaque
data:
authToken: <AUTH_TOKEN_ENCODED>

Multiple Authorization Keys

You can configure multiple authorization tokens inside the secret. These tokens can contain any string value, and become a human-readable identifier to reference the AUTH_TOKEN.

Note that in the following example, we have both authkey1 and authkey2:

apiVersion: v1
kind: Secret
metadata:
name: kfuse-auth-ingest
type: Opaque
data:
authkey1: <AUTH_TOKEN_ENCODED>
authkey2: <AUTH_TOKEN_ENCODED2>

Update the custom-values.yaml file to enable ingestion authentication:

global:
authConfig:
    enabled: true

Additional Labels Based on AUTH_TOKEN

You can configure additional labels to attach to ingested time data, and base these labels on the auth token of the incoming payload. To configure additional labels, add the following settings to the custom-values.yaml file. Note that Kloudfuse uses the same keys in the secret and in the ingester configuration:

ingester:
config:
     authKeyAdditionalLabels:
        authkey1:
          - name: label1
            value: val1
          - name: label2
            value: val2
        authkey2:
          - name: label1
            value: val3
          - name: label4
            value: val4

Restart the Ingester

After saving the configuration, restart the ingester service:

kubectl rollout restart statefulset ingester