# GCP Logs Collection

## Table of Contents

- [Create a GCP Subscription](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#subscription)
- [Configure Service Account for Pub/Sub Access](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#service-account)
  - [Assign Pub/Sub Subscriber Role](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#assign-pub-sub-subscriber-role)
  - [Create the Service Account Key](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#create-service-account-key)
  - [Create Kubernetes Secret](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#create-kubernetes-secret)
- [Configure Kloudfuse to Consume Logs from the GCP Subscription](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#consume)

Kloudfuse integrates with the GCP cloud service to collect logs through the Pub/Sub console.

Complete these tasks to successfully collect logs:

- [Create a GCP Subscription](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#subscription)
- [Configure Service Account for Pub/Sub Access](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#service-account)
- [Configure Kloudfuse to Consume Logs from the GCP Subscription](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/logs/#consume)

## Create a GCP Subscription

01. Use an existing project, or create a new project in [Cloud Pub-Sub console](https://console.cloud.google.com/cloudpubsub)

Create New Project

02. In the [Cloud Pub-Sub console](https://console.cloud.google.com/cloudpubsub/topicList), create a new topic.

Create Topic

Specify Topic

03. In the **Pub/Sub Topics** overview page, select **Subscriptions** in the left-hand navigation.

04. Click **Create Subscription**.

Create Subscription

05. Create a subscription with ID `kloudfuse-gcp-subscription` and select the topic you previously created, `MyLogsCollector`.

Click **Create**.

Specify Subscription

06. Confirm that the new subscription exists.

New Subscription created successfully

07. In the [Logs Explorer page](https://console.cloud.google.com/logs/viewer) of the console, under **More actions**, choose **Create sink** from the drop-down.

Create Sink

08. In **Sink details**, specify the **Name** and optional **Description** of the sink.

Click **Next**.

Sink details

09. In **Sink destination**, in **Select sink service**, choose **_Cloud Pub/Sub_**. In **Select a Cloud Pub/Sub Topic**, select the topic that you created in a previous step.

Click **Next**.

Sink destination

10. In the **Choose logs to include in sink**, create optional inclusion filters for the logs.

Click **Preview logs** to see the available logs.

Click **Next**.

Preview Logs

11. In the **Choose logs to filter out of sink**, create optional exclusion filters for the logs.

Click **Next**.

12. Click **Create sink**.

Create sink

13. The confirmation message appears.

Confirmation

## Configure Service Account for Pub/Sub Access

To allow Kloudfuse to consume logs from the GCP Pub/Sub subscription, you must configure a service account with the appropriate permissions and generate credentials.

### Assign Pub/Sub Subscriber Role

1. In the **Google Cloud** console, navigate to **Menu > IAM & Admin > Service Accounts**.

2. Select your service account, or create a new one.

3. Navigate to **Menu > IAM & Admin > IAM**.

4. Click **Grant Access** to add a new principal, or edit an existing principal.

5. In the **New principals** field, enter the service account email.

6. In the **Assign roles** section, select **Pub/Sub Subscriber** (`roles/pubsub.subscriber`).

This role allows the service account to consume messages from the Pub/Sub subscription.

7. Click **Save**.

### Create the Service Account Key

1. In the **Google Cloud** console, navigate to **Menu > IAM & Admin > Service Accounts**.

2. Select the service account you configured with the Pub/Sub Subscriber role.

3. Select **Keys > Add key > Create new key**.

4. Select **JSON**, then click **Create**.

The JSON key file downloads to your local machine.

5. Click **Close**.

### Create Kubernetes Secret

|     |     |
| --- | --- |
|  | The credentials file must be named `credentials.json` when creating the Kubernetes secret. |

Create a Kubernetes secret from the downloaded JSON key file:

```console
kubectl create secret generic kfuse-gcp-credentials \
  --from-file=credentials.json=<your-service-account-key>.json \
  -n kfuse
```

Copied!

|     |     |
| --- | --- |
|  | This is the same secret used by [GCP Metrics Collection](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/gcp/metrics/#secret). If you already created it for metrics, reuse it here — just ensure the service account also has the `roles/pubsub.subscriber` role. |

## Configure Kloudfuse to Consume Logs from the GCP Subscription

Add the following configuration to the `custom-values.yaml` file when installing Kloudfuse using Helm.

```yaml
global:
  ...
  enrichmentEnabled:
    - gcp
  gcpConfig:
    secretName: "kfuse-gcp-credentials" (1)
    pubsub:
      enabled: true
      subscriptionId: "kloudfuse-gcp-subscription" (2)
  ...
```

yamlCopied!

|     |     |
| --- | --- |
| **1** | The name of the Kubernetes secret containing the GCP service account credentials. |
| **2** | The Pub/Sub subscription ID created in the previous section. |