Heroku Log Ingestion :: Kloudfuse Docs

Heroku Log Ingestion

Kloudfuse integrates with Heroku to collect and ingest logs from Heroku applications using HTTPS log drains. Heroku’s log draining feature allows you to stream application and system logs to Kloudfuse for centralized monitoring and analysis.

For more information on Heroku logging, see Heroku Log Drains.

Prerequisites

Heroku application deployed and running
Heroku CLI installed and authenticated
Kloudfuse instance accessible via HTTPS
Kloudfuse ingress endpoint URL

Get Kloudfuse Endpoint

Obtain the Kloudfuse ingress endpoint URL.


	Use the external ingress IP or DNS name.

Note the endpoint URL format for Heroku log drains:

   https://<kfuse-ingress-external-ip>/ingester/heroku/logs

Configure Heroku Log Drain

Using Heroku CLI

Add the Kloudfuse log drain to your Heroku application:

heroku drains:add "https://<kfuse-ingress-external-ip>/ingester/heroku/logs" --app <app-name>

Replace:

<kfuse-ingress-external-ip> with your Kloudfuse endpoint
<app-name> with your Heroku application name

Verify the log drain was created successfully:
```
heroku drains --app <app-name>
```
You should see output similar to:
```
https://<kfuse-ingress-external-ip>/ingester/heroku/logs (d.12345678-90ab-cdef-1234-567890abcdef)
```
The drain ID specified in () will be added as a label in kloudfuse for easier filtering and organization.

Authentication

For information on configuring ingestion authentication, see Ingestion Authentication with API Key.

If authentication is enabled, append the encoded API key or token to the drain URL with the label kf-api-key:

heroku drains:add "https://<kfuse-ingress-external-ip>/ingester/heroku/logs?kf-api-key=YOUR_API_KEY" --app <app-name>

Log Format and Metadata

Heroku sends logs in the Logplex format. Kloudfuse automatically extracts metadata from Heroku logs, including:

Field	Description
`source`	source for all logs will be labeled as `Heroku`
`drain_token`	Drain token (e.g., `d.12345678-90ab-cdef-1234-567890abcdef`)
`timestamp`	Log timestamp (e.g., `2022-01-01T00:00:00.000Z`)
`log_level`	Log level (e.g., `info`, `error`)
`log_type`	Log type (e.g., `heroku`, `app`)

Configuration

Add Custom Labels

To add custom labels for filtering and organization, you can add them as parameters to the drain URL:

heroku drains:add "https://<kfuse-ingress-external-ip>/ingester/heroku/logs?label1=value1&label2=value2" --app <app-name>

You can use both custom labels and Authentication API keys in the parameters. If the API key parameter is present, it will not be added as a label.

Troubleshooting

Logs Not Appearing

Verify the log drain is active:
```
heroku drains --app <app-name>
```
Auth Config If Auth Config is enabled, confirm that ingest secret is base 64 encoded, and that the param label is kf-api-key
Review Heroku logs for drain errors:
```
heroku logs --tail --app <app-name>
```

Authentication Failures

Verify your API key is valid and has ingestion permissions.
Ensure the API key is properly URL-encoded in the drain URL.
Check Kloudfuse authentication configuration.

Remove Log Drain

To remove the Kloudfuse log drain from your Heroku application use the drain URL from the add step:

heroku drains:remove "https://<kfuse-ingress-external-ip>/ingester/heroku/logs" --app <app-name>

You can also view all drain URLs for a Heroku application via the console:

heroku drains --app <app-name>