Azure Integration Architecture :: Kloudfuse Docs
Azure Integration Architecture
Kloudfuse supports two parallel paths for ingesting Azure observability data: a log pipeline built on Azure Event Hubs and Azure Functions, and a metrics pipeline powered by the Kloudfuse Cloud Exporter.
Overview
Azure integration uses two independent pipelines that can be deployed together or separately:
Log pipeline — Azure resources emit logs to an Event Hub via Diagnostic Settings. An Azure Function App reads from the Event Hub and forwards records to the Kloudfuse HTTP ingestion endpoint.
Metrics pipeline — The Kloudfuse Cloud Exporter, running inside your AKS cluster, periodically scrapes the Azure Monitor Metrics API and stores the results in Kloudfuse.
Both pipelines write to the same Kloudfuse Storage and Query Layer, so logs and metrics from the same Azure resource appear together in the Kloudfuse UI.
Log Collection Pipeline
Data flows as follows:
- Azure resources (VMs, App Services, databases, and others) emit activity logs and resource logs.
- Diagnostic Settings route those logs to an Azure Event Hub Namespace.
- An Azure Function App with an Event Hub trigger reads each event and forwards it to the Kloudfuse logs ingestion endpoint over HTTPS.
- Kloudfuse stores the logs and makes them searchable in Log Analytics.
This pipeline is event-driven: logs arrive in Kloudfuse within seconds of being emitted by the Azure resource. The Function App handles batching and retries automatically.
Metrics Collection Pipeline
The Kloudfuse Cloud Exporter is a Prometheus-compatible scraper that runs as a pod in your AKS cluster alongside Kloudfuse. It calls the Azure Monitor Metrics API on a configurable interval (default: five minutes) to retrieve metric data points for all resources in the specified subscriptions.
Data flows as follows:
- Azure resources publish metrics to Azure Monitor.
- The Cloud Exporter polls the Azure Monitor Metrics API and retrieves metric data.
- The Cloud Exporter writes the metrics directly into the Kloudfuse storage layer.
Because the Cloud Exporter runs inside your cluster, no inbound firewall rules are required. Only outbound HTTPS access to the Azure Monitor Metrics API endpoint (management.azure.com) is needed.
Authentication
Both pipelines authenticate to Azure using a Service Principal (app registration):
| Credential | Where it is used |
|---|---|
Client ID (AZURE_CLIENT_ID) |
Identifies the app registration |
Tenant ID (AZURE_TENANT_ID) |
Identifies the Azure Active Directory tenant |
Client Secret (AZURE_CLIENT_SECRET) |
Authenticates the app registration |
| Subscription ID | Scopes which Azure subscriptions to collect metrics from |
The Service Principal must be assigned the Monitoring Reader role on each subscription you want to monitor. For the log pipeline, the Function App authenticates to the Event Hub using a connection string (SAS policy).
Choosing an Integration Method
| Requirement | Use |
|---|---|
| Collect Azure activity and resource logs | Azure Log Integration |
| Collect Azure Monitor metrics | Azure Metrics Integration |
| Full Azure observability (logs and metrics) | Both pipelines |
Next Steps
- Azure Log Integration — Set up the Event Hub and Function App log pipeline.
- Azure Metrics Integration — Configure the Cloud Exporter for Azure Monitor metrics.