Quick Guide for OTel Agent on Kubernetes for Infrastructure Objects :: Kloudfuse Docs

Quick Guide for OTel Agent on Kubernetes for Infrastructure Objects

This guide shows an end-to-end workflow to collect, process, and export Kubernetes infrastructure objects (pods, nodes, deployments, etc.) to Kloudfuse using OpenTelemetry Collector. You will configure receivers, processors, pipelines, exporters, and RBAC, then verify data in the Kloudfuse UI.

Workflow Overview

The data collection flow is:

Kubernetes Objects → k8sobjects Receiver → Processors (k8sattributes, resource, resourcedetection) → Dedicated Logs Pipeline → OTLPHTTP Exporter → Kloudfuse UI

Steps:

Install OpenTelemetry Collector in your Kubernetes cluster.
Configure k8sobjects receiver to capture objects.
Apply processors for enrichment and metadata extraction.
Define a dedicated logs pipeline for infrastructure objects.
Configure otlphttp exporter to send data to Kloudfuse.
Apply RBAC permissions to allow object access.
Enable leader election to avoid duplicate collection.
Verify ingestion in the Kloudfuse UI.

Key Points

Each data type (logs, events, infrastructure objects) requires a dedicated pipeline.
Exporter names must be prefixed with otlphttp/ (e.g., otlphttp/k8sobjects).
Use the fixed ingestion endpoint: https://<KLOUDFUSE_ADDRESS>/ingester/otlp/k8s_objects.
Enrich objects with k8sattributes, resource, and resourcedetection processors.
Ensure RBAC permissions cover all object types collected.
Enable leader election for multi-Collector deployments.
Enable kubernetes and system-level metrics collection. (See Kubernetes and System-Level Metrics)


	- Do not reuse existing pipelines. Each pipeline must handle only one data type. - Misconfigured exporter names or endpoints will prevent ingestion. - All collected object types must be covered in RBAC rules.

Step 1: Configure Exporter

Use an OTLP HTTP exporter for infrastructure objects.

exporters:
  otlphttp/k8sobjects:
    logs_endpoint: https://<KLOUDFUSE_ADDRESS>/ingester/otlp/k8s_objects

Step 2: Configure Receiver

Add the k8sobjects receiver to collect desired Kubernetes objects.


	By default, Kloudfuse automatically drops any object that has not been updated within 15 minutes. To prevent this, you can configure each receiver object with a custom update interval, such as: `yaml<br>- name: pods<br> interval: 5m<br> mode: pull<br>`

receivers:
  k8sobjects:
    objects:
      - name: pods
      - name: deployments
      - name: replicasets
      - name: services
      - name: nodes
      - name: jobs
      - name: cronjobs
      - name: daemonsets
      - name: statefulsets
      - name: persistentvolumes
      - name: persistentvolumeclaims
      - name: roles
      - name: rolebindings
      - name: clusterroles
      - name: clusterrolebindings
      - name: serviceaccounts
      - name: ingresses
      - name: namespaces

Example:

For lightweight testing, collect only pods and deployments:

receivers:
  k8sobjects:
    objects:
      - name: pods
      - name: deployments

Step 3: Configure Processors

Use processors to enrich objects and prepare them for Kloudfuse dashboards.

processors:
  resource:
    attributes:
      - key: kf_infra_agent
        value: "otlp"
        action: upsert   # required for filtering in Kloudfuse UI

resourcedetection:
    detectors: [env, gcp]  # replace with your environment (eks, aks, ec2, azure)
    timeout: 2s
    override: false

k8sattributes:
    extract:
      metadata:
        - k8s.namespace.name
        - k8s.deployment.name
        - k8s.statefulset.name
        - k8s.daemonset.name
        - k8s.cronjob.name
        - k8s.job.name
        - k8s.node.name
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.pod.start_time
        - k8s.replicaset.name

Example:

Filter pods and annotate them for dashboards:

processors:
  resource:
    attributes:
      - key: kf_infra_agent
        value: "otlp"
        action: upsert

Step 4: Configure Pipeline

Define a dedicated logs pipeline for infrastructure objects.

service:
  pipelines:
    logs/k8sobjects:
      receivers: [k8sobjects]
      processors: [k8sattributes, resource, resourcedetection]
      exporters: [otlphttp/k8sobjects]

Step 5: Configure RBAC Permissions

Grant the Collector access to all object types.

rules:
- apiGroups: [""]
  resources: [namespaces, nodes, pods, services, persistentvolumeclaims, persistentvolumes, serviceaccounts]
  verbs: ["get", "list", "watch"]

- apiGroups: ["apps"]
  resources: [daemonsets, deployments, replicasets, statefulsets]
  verbs: ["get", "list", "watch"]

- apiGroups: ["rbac.authorization.k8s.io"]
  resources: [clusterroles, clusterrolebindings, roles, rolebindings]
  verbs: ["get", "list", "watch"]

- apiGroups: ["networking.k8s.io"]
  resources: [ingresses]
  verbs: ["get", "list", "watch"]

- apiGroups: ["batch"]
  resources: [jobs, cronjobs]
  verbs: ["get", "list", "watch"]

Step 6: Enable Leader Election

Prevent duplicate collection in multi-Collector deployments.

extensions:
  leader_election: {}

service:
  extensions: [leader_election]

Step 7: Complete YAML Example

Combines receivers, processors, exporters, and pipelines.

extensions:
  leader_election: {}

receivers:
  k8sobjects:
    objects:
      - name: pods
      - name: deployments
      - name: replicasets
      - name: services
      - name: nodes
      - name: jobs
      - name: cronjobs
      - name: daemonsets
      - name: statefulsets
      - name: persistentvolumes
      - name: persistentvolumeclaims
      - name: roles
      - name: rolebindings
      - name: clusterroles
      - name: clusterrolebindings
      - name: serviceaccounts
      - name: ingresses
      - name: namespaces

processors:
  resource:
    attributes:
      - key: kf_infra_agent
        value: "otlp"
        action: upsert
  resourcedetection:
    detectors: [env, gcp]
    timeout: 2s
    override: false
  k8sattributes:
    extract:
      metadata:
        - k8s.namespace.name
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.node.name
        - k8s.deployment.name
        - k8s.replicaset.name
        - k8s.statefulset.name
        - k8s.daemonset.name
        - k8s.job.name
        - k8s.cronjob.name

exporters:
  otlphttp/k8sobjects:
    logs_endpoint: https://<KLOUDFUSE_ADDRESS>/ingester/otlp/k8s_objects

service:
  extensions: [leader_election]
  pipelines:
    logs/k8sobjects:
      receivers: [k8sobjects]
      processors: [k8sattributes, resource, resourcedetection]
      exporters: [otlphttp/k8sobjects]

Step 8: Verify Collection

Restart the OTel Collector.
Open Kloudfuse UI: Infrastructure > Kubernetes.
Confirm that pods, nodes, deployments, and namespaces appear.
Use filter kf_infra_agent=otlp to view only OTEL-sourced objects.