AWS Lambda Integration with Kloudfuse

Kloudfuse integrates with AWS Lambda to collect metrics, logs, and traces from your serverless functions.

Add Extension Layers to the Lambda Function

Kloudfuse requires that you configure two extension layers for the lambda function.

Add these layers in the AWS Lambda console of the Lambda function:

• Datadog Extension Layer
  Kloudfuse tested and verified with version 33.

Layer ARN:

`arn:aws:lambda:us-west-2:464622532012:layer:Datadog-Extension:33`

• LambdaInsightsExtension
  Kloudfuse tested and verified with version 21.

Layer ARN:

arn:aws:lambda:us-west-2:580247275435:layer:LambdaInsightsExtension:21

Configure the Datadog Extension Layer

Add the following environment variables to the Lambda configuration:

• DD_API_KEY
  If authenticated ingest is enabled, provide the configured auth token. Else, provide any string.

• DD_APM_DD_URL
  https://<external facing endpoint of Kfuse cluster>/ingester

• DD_DD_URL
  https://<external facing endpoint of Kfuse cluster>/ingester

• DD_LOGS_CONFIG_LOGS_DD_URL
  <external facing endpoint of Kfuse cluster>:443

• DD_LOGS_CONFIG_LOGS_NO_SSL
  false

• DD_LOGS_CONFIG_USE_V2_API
  false

• DD_TRACE_ENABLED
  true

Configure Cloudwatch Metrics

After you integrate CloudWatch with Kloudfuse, it pushes Lambda-related metrics by default.

See Cloudwatch Metrics.

Configure CloudTrail

Configure CloudTrail to send Lambda events to the EventBridge, and then onward to Kloudfuse.

1. Create a new Trail from the AWS CloudTrail console.

2. In Step 2 Choose log events, make these selections:

• Event type: Data events
  • Data event type: Lambda
  • Log selector template: Log all events

Update Helm Values in Kloudfuse

1. Enable Lambda enrichment in custom_values.yaml file.

ingester:
     config:
       awsScrapeLambdaConfigs: true

2. Kloudfuse must scrape the Lambda configuration from AWS, so it must have a policy with following permissions:

{
       "Action": [\
           {\
               "lambda:GetPolicy",\
               "lambda:List*",\
               "lambda:ListTags"\
           }\
       ]
}

Ensure that the permissions map to the node-pool used for EKS cluster that hosts the Kloudfuse platform.

3. Create an IAM scraper role with a policy that enables scraping on AWS labels.

See AWS documentation on how to Define custom IAM permissions with customer managed policies.

4. Enable Kloudfuse to consume the new policy; there are two approaches: through AWS credentials or through Role ARNs.

AWS credentials

1. Retrieve your aws credentials; see Configure tool authentication with AWS.

2. In the Kloudfuse namespace, create a kube secret name aws-access-key, with keys accessKey and secretKey.

kubectl create secret generic aws-access-key --from-literal=accessKey=<AWS_ACCESS_KEY_ID> --from-literal=secretKey=<AWS_SECRET_ACCESS_KEY>

3. Specify the secretName in the custom-values.yaml file.

ingester:
  config:
    awsScraper:
      secretName: aws-access-key

4. By default, Kloudfuse attempts to scrape from all regions. Customize this by adding the following configuration in the custom-values.yaml file:

ingester:
  config:
    awsScraper:
      secretName: aws-access-key
      regions:
    - <add region>

Role ARNs

1. Add the scraper Role ARNs that you created with the new permissions to the awsRoleArns list to your custom-values.yaml file.

ingester:
  config:
    awsRoleArns:
    - role: <ADD ROLE ARN HERE>

2. By default, Kloudfuse attempts to scrape from all regions. Customize this by adding the following configuration in the custom-values.yaml file:

ingester:
  config:
    awsRoleArns:
      role: <ADD ROLE ARN HERE>
      regions:
    - <add region>

3. You may have to modify the Trust Relationship for the policy of the scrape role ARN, to add a Principal on the account for the node-group (Node IAM Role ARN) where the Kloudfuse platform runs.

{
    "Version": "2012-10-17",
    "Statement": [\
        {\
            "Sid": "Statement1",\
            "Effect": "Allow",\
            "Principal": {\
                "AWS": "arn:aws:iam::ACCOUNT-NUMBER:role/eksctl-XXXXX-nodegroup-ng-XXXXXX-NodeInstanceRole-XXXXXXXXXX"\
            },\
            "Action": "sts:AssumeRole"\
        }\
    ]
}

4. To modify the node-group IAM role where Kloudfuse Platform runs, add the following permissions policy to the node-group (Node IAM Role ARN) to assume the role.

{
    "Version": "2012-10-17",
    "Statement": [\
        {\
            "Effect": "Allow",\
            "Action": "sts:AssumeRole",\
            "Resource": <REPLACE SCRAPER ROLE ARN HERE>\
        }\
    ]
}

5. Complete a helm upgrade to save the changes.

helm upgrade --create-namespace --install kfuse . -f <custom_values.yaml>

Configure EventBridge

After you integrate EventBridge with Kloudfuse, it pushes Lambda-related events to Kloudfuse.

See AWS EventBridge Integration with Kloudfuse.