# GCP Logs Collection using Pub/Sub

Kloudfuse integrates with the GCP cloud service to collect logs through the Pub/Sub console.

Complete these tasks to successfully collect logs:

- [Create a GCP Subscription](https://docs.kloudfuse.com/platform/3.5.0/cloud-service-gcp-logs/#subscription)
- [Configure Service Account for Pub/Sub Access](https://docs.kloudfuse.com/platform/3.5.0/cloud-service-gcp-logs/#service-account)
- [Configure Kloudfuse to Consume Logs from the GCP Subscription](https://docs.kloudfuse.com/platform/3.5.0/cloud-service-gcp-logs/#consume)

## Create a GCP Subscription

01. Use an existing project, or create a new project in [Cloud Pub-Sub console](https://console.cloud.google.com/cloudpubsub)

Create New Project

02. In the [Cloud Pub-Sub console](https://console.cloud.google.com/cloudpubsub/topicList), create a new topic.

Create Topic

Specify Topic

03. In the **Pub/Sub Topics** overview page, select **Subscriptions** in the left-hand navigation.

04. Click **Create Subscription**.

Create Subscription

05. Create a subscription with ID `kloudfuse-gcp-subscription` and select the topic you previously created, `MyLogsCollector`.

Click **Create**.

Specify Subscription

06. Confirm that the new subscription exists.

New Subscription created successfully

07. In the [Logs Explorer page](https://console.cloud.google.com/logs/viewer) of the console, under **More actions**, choose **Create sink** from the drop-down.

Create Sink

08. In **Sink details**, specify the **Name** and optional **Description** of the sink.

Click **Next**.

Sink details

09. In **Sink destination**, in **Select sink service**, choose **_Cloud Pub/Sub_**. In **Select a Cloud Pub/Sub Topic**, select the topic that you created in a previous step.

Click **Next**.

Sink destination

10. In the **Choose logs to include in sink**, create optional inclusion filters for the logs.

Click **Preview logs** to see the available logs.

Click **Next**.

Preview Logs

11. In the **Choose logs to filter out of sink**, create optional exclusion filters for the logs.

Click **Next**.

12. Click **Create sink**.

Create sink

13. The confirmation message appears.

Confirmation

## Configure Service Account for Pub/Sub Access

To allow Kloudfuse to consume logs from the GCP Pub/Sub subscription, you must configure a service account with the appropriate permissions and generate credentials.

### Assign Pub/Sub Subscriber Role

1. In the **Google Cloud** console, navigate to **Menu > IAM & Admin > Service Accounts**.
2. Select your service account, or create a new one.
3. Navigate to **Menu > IAM & Admin > IAM**.
4. Click **Grant Access** to add a new principal, or edit an existing principal.
5. In the **New principals** field, enter the service account email.
6. In the **Assign roles** section, select **Pub/Sub Subscriber** (`roles/pubsub.subscriber`).

This role allows the service account to consume messages from the Pub/Sub subscription.

7. Click **Save**.

### Create and Encode the Service Account Key

1. In the **Google Cloud** console, navigate to **Menu > IAM & Admin > Service Accounts**.
2. Select the service account you configured with the Pub/Sub Subscriber role.
3. Select **Keys > Add key > Create new key**.
4. Select **JSON**, then click **Create**.

The JSON key file downloads to your local machine.

5. Click **Close**.
6. Base64-encode the JSON key file:

```console
cat <your-service-account-key>.json | base64
```

Copied!

7. Copy the base64-encoded output. You will use this value for the `pubsubKey` field in the Kloudfuse configuration.

## Configure Kloudfuse to Consume Logs from the GCP Subscription

Add the following configuration to the `custom-values.yaml` file when installing Kloudfuse using Helm, and add the GCP pub/sub access key in the `pubsubKey` field.

```yaml
global:
  ...
  enrichmentEnabled:
    - gcp
  gcpConfig:
    enabled: true
    subscriptionId: "kloudfuse-gcp-subscription" (1)
    pubsubKey: "<BASE64_ENCODED_SERVICE_ACCOUNT_KEY>" (2)
  ...
```
yamlCopied!

|     |     |
| --- | --- |
| **1** | Add the subscription information. |
| **2** | Paste the base64-encoded service account JSON key from the previous section.