AWS CloudTrail Events Integration

Kloudfuse ingests AWS CloudTrail events by routing them through CloudWatch Logs and then to a Kinesis Firehose delivery stream. This gives you a searchable record of every API call made in your AWS account alongside your metrics and application logs.

Overview

AWS CloudTrail records every API call made in your AWS account — including calls from the AWS Console, CLI, SDKs, and other AWS services. Examples of events captured include: EC2 instance launches, IAM policy changes, S3 object deletions, Lambda invocations, and login activity.

Common use cases for CloudTrail in Kloudfuse:

• Security monitoring — detect unusual API patterns, unauthorized access attempts, or privilege escalation.
• Compliance reporting — produce an audit trail showing who changed what, and when.
• Incident investigation — correlate infrastructure changes with service disruptions using Kloudfuse’s unified log and metrics view.
• Change tracking — track configuration changes to AWS resources over time.

Data flows as follows:

1. CloudTrail records API events and sends them to an S3 bucket (default) and optionally to a CloudWatch Logs log group.
2. A Kinesis Firehose subscription filter on the CloudWatch log group forwards records to Kloudfuse.

Prerequisites

• A Kinesis Firehose delivery stream named kloudfuse-logs and its associated IAM role. If you have not created this yet, complete Configure AWS Kinesis Firehose for Kloudfuse first.
• An S3 bucket to receive CloudTrail event archives.
• AWS CLI configured with permissions to manage CloudTrail trails, CloudWatch Logs, and IAM.

Step 1: Create a CloudTrail Trail

If you do not already have a trail configured, create one. A trail must be configured to deliver events to a CloudWatch Logs log group to forward them to Kloudfuse.

First, create an S3 bucket for CloudTrail (if needed):

aws s3api create-bucket \
  --bucket my-cloudtrail-bucket \
  --region us-west-2 \
  --create-bucket-configuration LocationConstraint=us-west-2

Apply a bucket policy that allows CloudTrail to write to it:

cat > cloudtrail-bucket-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [\
    {\
      "Sid": "AWSCloudTrailAclCheck",\
      "Effect": "Allow",\
      "Principal": {"Service": "cloudtrail.amazonaws.com"},\
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::my-cloudtrail-bucket"\
    },\
    {\
      "Sid": "AWSCloudTrailWrite",\
      "Effect": "Allow",\
      "Principal": {"Service": "cloudtrail.amazonaws.com"},\
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-cloudtrail-bucket/AWSLogs/*",\
      "Condition": {"StringEquals": {"s3:x-amz-acl": "bucket-owner-full-control"}}\
    }\
  ]
}
EOF

aws s3api put-bucket-policy \
  --bucket my-cloudtrail-bucket \
  --policy file://cloudtrail-bucket-policy.json

Create the trail:

aws cloudtrail create-trail \
  --name kloudfuse-trail \
  --s3-bucket-name my-cloudtrail-bucket \
  --include-global-service-events \
  --is-multi-region-trail

aws cloudtrail start-logging \
  --name kloudfuse-trail

Step 2: Send CloudTrail Events to CloudWatch Logs

Create the IAM role that allows CloudTrail to write to CloudWatch Logs:

cat > cloudtrail-cw-trust.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [{\
    "Effect": "Allow",\
    "Principal": {"Service": "cloudtrail.amazonaws.com"},\
    "Action": "sts:AssumeRole"\
  }]
}
EOF

aws iam create-role \
  --role-name CloudTrailToCloudWatchRole \
  --assume-role-policy-document file://cloudtrail-cw-trust.json

aws iam put-role-policy \
  --role-name CloudTrailToCloudWatchRole \
  --policy-name CloudTrailLogsPolicy \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{\
      "Effect": "Allow",
      "Action": ["logs:CreateLogStream", "logs:PutLogEvents"],
      "Resource": "arn:aws:logs:*:*:log-group:*"
    }]
  }'

Create the CloudWatch log group and configure the trail to use it:

aws logs create-log-group --log-group-name /aws/cloudtrail/kloudfuse

ROLE_ARN=$(aws iam get-role \
  --role-name CloudTrailToCloudWatchRole \
  --query 'Role.Arn' \
  --output text)

LOG_GROUP_ARN=$(aws logs describe-log-groups \
  --log-group-name-prefix /aws/cloudtrail/kloudfuse \
  --query 'logGroups[0].arn' \
  --output text)

aws cloudtrail update-trail \
  --name kloudfuse-trail \
  --cloud-watch-logs-log-group-arn "$LOG_GROUP_ARN" \
  --cloud-watch-logs-role-arn "$ROLE_ARN"

Step 3: Subscribe to Kinesis Firehose

Subscribe the CloudTrail log group to the kloudfuse-logs Firehose delivery stream:

aws logs put-subscription-filter \
  --log-group-name /aws/cloudtrail/kloudfuse \
  --filter-name kloudfuse \
  --filter-pattern "" \
  --destination-arn "arn:aws:firehose:<region>:<account-id>:deliverystream/kloudfuse-logs" \
  --role-arn "arn:aws:iam::<account-id>:role/KloudfuseFirehoseRole"

Enable Data Events (Optional)

By default, CloudTrail only captures management events. Data events (S3 object-level operations, Lambda invocations) must be explicitly enabled and are billed separately.

Enable Lambda data events:

aws cloudtrail put-event-selectors \
  --trail-name kloudfuse-trail \
  --event-selectors '[\
    {\
      "ReadWriteType": "All",\
      "IncludeManagementEvents": true,\
      "DataResources": [\
        {\
          "Type": "AWS::Lambda::Function",\
          "Values": ["arn:aws:lambda"]\
        }\
      ]\
    }\
  ]'

Enable S3 data events for a specific bucket:

aws cloudtrail put-event-selectors \
  --trail-name kloudfuse-trail \
  --event-selectors '[\
    {\
      "ReadWriteType": "All",\
      "IncludeManagementEvents": true,\
      "DataResources": [\
        {\
          "Type": "AWS::S3::Object",\
          "Values": ["arn:aws:s3:::my-bucket/"]\
        }\
      ]\
    }\
  ]'

Verify the Integration

Confirm Events are being Sent

1. Confirm the trail is actively logging:

aws cloudtrail get-trail-status \
     --name kloudfuse-trail \
     --query '{IsLogging: IsLogging, LatestDeliveryTime: LatestDeliveryTime}'

IsLogging must be true. LatestDeliveryTime should be recent (within the last few minutes).

2. Confirm the CloudWatch log group subscription filter points to the correct Firehose stream:

aws logs describe-subscription-filters \
     --log-group-name /aws/cloudtrail/kloudfuse \
     --query 'subscriptionFilters[*].{Filter:filterName,Destination:destinationArn}'

The destinationArn must reference the kloudfuse-logs delivery stream.

3. In the AWS Console, open Amazon Kinesis and select the kloudfuse-logs delivery stream.
4. Click the Monitoring tab and check the IncomingRecords and DeliveryToHTTPEndpoint.Success graphs. Both should show non-zero values within a few minutes of making an API call in your account.
5. If DeliveryToHTTPEndpoint.Success is zero, check the DeliveryToHTTPEndpoint.DataFreshness metric for errors and verify the Kloudfuse ingester endpoint URL and API key in the Firehose HTTP endpoint configuration.

Confirm Events are Arriving in Kloudfuse

1. Make an API call in your AWS account (for example, run any aws CLI command) to generate a fresh event.
2. In the Kloudfuse UI, click the Logs tab and select Search from the drop-down.
3. Set the time picker to the last 15 minutes.
4. In the search bar, click Advanced Search and enter the following FuseQL query to confirm events are flowing:

eventSource="cloudtrail.amazonaws.com" or eventVersion="1.08"

Any result confirms that CloudTrail JSON records are reaching Kloudfuse.

5. To narrow the results to a specific AWS service, filter by eventSource. For example, to view IAM activity:

eventSource="iam.amazonaws.com" | count by eventName

6. To find console sign-in events:

eventName="ConsoleLogin"

7. To find events from a specific IAM user or role:

eventSource="iam.amazonaws.com" and userIdentity.type="IAMUser"