Audit Images from ECR :: Kloudfuse Docs

Audit Images from ECR

This guide explains how to configure OpenID Connect (OIDC) authentication to download Kloudfuse container images from AWS ECR and verify their signatures in CI/CD pipelines without managing long-lived AWS credentials.

Overview

When downloading Kloudfuse container images in CI/CD pipelines, you need AWS ECR access to pull images and AWS KMS access to verify their signatures. Instead of storing AWS credentials (access keys) in your CI/CD environment, you can use OIDC to establish a trust relationship between your CI/CD provider (like GitLab) and AWS.

Benefits of OIDC Authentication

No long-lived credentials - No AWS access keys to manage or rotate
Automatic token generation - CI/CD provider generates short-lived tokens automatically
Fine-grained access control - Limit access to specific projects, branches, or jobs
Enhanced security - Tokens expire automatically and cannot be reused
Audit trail - AWS CloudTrail logs show which jobs assumed roles

Architecture

The OIDC authentication flow works as follows:

Setup Process

Setting up OIDC authentication requires configuring both AWS and your CI/CD provider.

Step 1: Create OIDC Provider in AWS

First, establish trust between AWS and your CI/CD platform’s OIDC provider.

For GitLab (Self-Hosted or GitLab.com)

Navigate to AWS IAM Console → Identity Providers → Add Provider
Configure the OIDC provider:
- Provider Type: OpenID Connect
- Provider URL: Your GitLab OIDC endpoint
- Audience: Your GitLab instance URL

Example for GitLab Corporate Instance:

Provider URL: https://gitlab-oidc-config.s3.us-west-2.amazonaws.com
Audience: https://gitlab.example.com

Example for GitLab.com:

Provider URL: https://gitlab.com
Audience: https://gitlab.com

Step 2: Create and Attach IAM Policies for ECR and KMS Access

Create and attach policies that grant permissions to pull Kloudfuse container images from ECR and verify their signatures.

For ECR Access:

Choose one of the following AWS managed ECR policies based on your needs:

AmazonEC2ContainerRegistryReadOnly (recommended)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:DescribeImages",
                "ecr:BatchGetImage",
                "ecr:GetLifecyclePolicy",
                "ecr:GetLifecyclePolicyPreview",
                "ecr:ListTagsForResource",
                "ecr:DescribeImageScanFindings"
            ],
            "Resource": "*"
        }
    ]
}

AmazonEC2ContainerRegistryPullOnly (minimum privilege)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchImportUpstreamImage"
            ],
            "Resource": "*"
        }
    ]
}

For KMS Access:

Create a custom policy for KMS signature verification:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "KMSSignatureVerification",
            "Effect": "Allow",
            "Action": [
                "kms:GetPublicKey",
                "kms:Verify",
                "kms:DescribeKey"
            ],
            "Resource": [
                "arn:aws:kms:us-west-2:502496443919:alias/kfuse-cosign-signing-key"
            ]
        }
    ]
}

To attach these policies:

Navigate to AWS IAM Console → Policies → Create Policy
Create the KMS policy above and name it: KloudfuseImageSignatureVerifyPolicy
When creating the role in Step 3, attach both policies:
- AmazonEC2ContainerRegistryReadOnly or AmazonEC2ContainerRegistryPullOnly (AWS managed — choose one)
- KloudfuseImageSignatureVerifyPolicy (your custom policy)

Step 3: Create IAM Role with OIDC Trust Relationship

Create an IAM role that can be assumed by your CI/CD jobs using OIDC tokens.

Navigate to AWS IAM Console → Roles → Create Role
Select Web Identity as the trusted entity type
Configure the trust relationship:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::YOUR_AWS_ACCOUNT_ID:oidc-provider/gitlab.example.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "gitlab.example.com:aud": "https://gitlab.example.com"
                },
                "StringLike": {
                    "gitlab.example.com:sub": "project_path:your-organization/*"
                }
            }
        }
    ]
}

Attach the policies created in Step 2 (AmazonEC2ContainerRegistryReadOnly or AmazonEC2ContainerRegistryPullOnly, and KloudfuseImageSignatureVerifyPolicy)
Name the role: KloudfuseImagePullRole
Note the Role ARN - you will need this in your CI/CD pipeline

Step 4: Configure CI/CD Pipeline

Configure your CI/CD pipeline to use the OIDC role for AWS authentication.

GitLab CI/CD Example

verify-kfuse-image:
  stage: verify
  image: docker:latest
  services:
    - docker:dind

variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    IMAGE_NAME: ui
    IMAGE_TAG: "0.1.0-b2274ac0"
    COSIGN_VERSION: "v3.0.3"
    KMS_KEY: "awskms:///arn:aws:kms:us-west-2:502496443919:alias/kfuse-cosign-signing-key"
    AWS_ROLE_ARN: "arn:aws:iam::YOUR_AWS_ACCOUNT_ID:role/KloudfuseImageVerificationRole"
    AWS_DEFAULT_REGION: "us-west-2"

id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.example.com

before_script:
    - apk add --no-cache curl aws-cli jq
    - curl -sSfL "https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/cosign-linux-amd64" -o /usr/local/bin/cosign
    - chmod +x /usr/local/bin/cosign
    - |
      echo "Assuming AWS role using OIDC..."

STS_RESPONSE=$(aws sts assume-role-with-web-identity \
        --role-arn "${AWS_ROLE_ARN}" \
        --role-session-name "gitlab-ci-${CI_PROJECT_NAME}-${CI_PIPELINE_ID}" \
        --web-identity-token "${GITLAB_OIDC_TOKEN}" \
        --duration-seconds 3600)

export AWS_ACCESS_KEY_ID=$(echo "${STS_RESPONSE}" | jq -r '.Credentials.AccessKeyId')
      export AWS_SECRET_ACCESS_KEY=$(echo "${STS_RESPONSE}" | jq -r '.Credentials.SecretAccessKey')
      export AWS_SESSION_TOKEN=$(echo "${STS_RESPONSE}" | jq -r '.Credentials.SessionToken')

echo "✅ Successfully authenticated to AWS"

script:
    - |
      IMAGE_REF="502496443919.dkr.ecr.us-west-2.amazonaws.com/kfuse/${IMAGE_NAME}:${IMAGE_TAG}"
      echo "Logging in to Kloudfuse ECR..."
      aws ecr get-login-password --region us-west-2 | \
        docker login --username AWS --password-stdin 502496443919.dkr.ecr.us-west-2.amazonaws.com

echo "Pulling image ${IMAGE_REF}..."
      docker pull "${IMAGE_REF}"

echo "Verifying signature for ${IMAGE_REF}..."
      if cosign verify --key "${KMS_KEY}" "${IMAGE_REF}"; then
        echo "✅ Image signature verified successfully"
      else
        echo "⚠️  Warning: Image signature verification FAILED"
      fi

echo "✅ Image pulled and verified successfully"

Verification and Testing

Test the Pipeline

Commit the pipeline configuration to your GitLab repository
Run the pipeline manually or via a commit
Check the job logs for:

Assuming AWS role using OIDC...
✅ Successfully authenticated to AWS
Logging in to Kloudfuse ECR...
Login Succeeded
Pulling image 502496443919.dkr.ecr.us-west-2.amazonaws.com/kfuse/ui:0.1.0-b2274ac0...
0.1.0-b2274ac0: Pulling from kfuse/ui
✅ Image pulled and verified successfully

Verify in AWS CloudTrail

Check AWS CloudTrail logs to confirm OIDC authentication:

Navigate to AWS CloudTrail Console → Event History
Filter by:
- Event name: AssumeRoleWithWebIdentity
- User name: Your role name
Inspect the event details to see:
- Which GitLab project assumed the role
- Session name (includes pipeline ID)
- Timestamp and source IP

Troubleshooting

"Not authorized to perform sts:AssumeRoleWithWebIdentity"
- Cause: OIDC provider not configured or trust relationship incorrect
- Solution: Verify the OIDC provider ARN in the role’s trust relationship matches the provider created in Step 1
"Access Denied" with audience mismatch
- Cause: The aud claim in the OIDC token doesn’t match the trust relationship
- Solution: Ensure id_tokens.GITLAB_OIDC_TOKEN.aud in your pipeline matches the audience in the trust relationship condition
"Access denied" with subject mismatch
- Cause: The sub claim in the OIDC token doesn’t match the trust relationship condition
- Solution: Verify the StringLike condition in the trust relationship matches your GitLab project path
"Token has expired"
- Cause: OIDC token lifetime exceeded
- Solution: OIDC tokens are typically valid for 5-10 minutes. Ensure assume-role-with-web-identity runs early in the job
"Access Denied" when calling KMS operations
- Cause: IAM role doesn’t have KMS permissions
- Solution: Verify the KMS policy from Step 2 is attached to the role

Debug OIDC Token Claims

To inspect what claims GitLab is sending in the OIDC token:

# Add this to your pipeline before_script to decode the token
echo "${GITLAB_OIDC_TOKEN}" | cut -d. -f2 | base64 -d 2>/dev/null | jq .

This will show claims like:

{
  "aud": "https://gitlab.example.com",
  "sub": "project_path:your-organization/your-project:ref:refs/heads/main",
  "iss": "https://gitlab.example.com",
  "exp": 1735123456,
  "iat": 1735123156,
  "jti": "abc123..."
}

Security Best Practices

Minimize trust scope - Use specific StringLike conditions to limit which projects/branches can assume the role
Short session duration - Use the minimum --duration-seconds needed for your job (typically 3600 = 1 hour)
Least privilege - Grant only the KMS permissions needed (GetPublicKey, Verify, DescribeKey)
Monitor CloudTrail - Set up alerts for unexpected AssumeRoleWithWebIdentity events
Review OIDC providers - Periodically audit and update OIDC provider configurations
Use session names - Include pipeline/job identifiers in session names for better audit trails

Advanced Configurations

Multiple AWS Accounts (Development, Staging, Production)

Use different IAM roles per environment:

.verify-template:
  stage: verify
  # ... (common configuration)
  script:
    - |
      case "${CI_ENVIRONMENT_NAME}" in
        production)
          AWS_ROLE_ARN="arn:aws:iam::111111111111:role/KfuseImageVerify-Prod"
          ;;
        staging)
          AWS_ROLE_ARN="arn:aws:iam::222222222222:role/KfuseImageVerify-Staging"
          ;;
        development)
          AWS_ROLE_ARN="arn:aws:iam::333333333333:role/KfuseImageVerify-Dev"
          ;;
      esac

# ... (assume role and verify)

verify-prod:
  extends: .verify-template
  environment: production
  only:
    - main

verify-staging:
  extends: .verify-template
  environment: staging
  only:
    - staging

Cross-Account KMS Access

If your AWS account differs from Kloudfuse’s KMS account (502496443919), you need to:

Ensure the KMS key policy allows your account
Add kms:* permissions for the key ARN in your IAM policy

Other CI/CD Providers

GitHub Actions:

- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::YOUR_ACCOUNT:role/KloudfuseImageVerificationRole
    aws-region: us-west-2

- name: Verify image signature
  run: |
    cosign verify \
      --key awskms:///arn:aws:kms:us-west-2:502496443919:alias/kfuse-cosign-signing-key \
      502496443919.dkr.ecr.us-west-2.amazonaws.com/kfuse/ui:0.1.0-b2274ac0

CircleCI: Use CircleCI’s OIDC token integration with AWS (see CircleCI OIDC docs)

Jenkins: Configure OIDC authentication using the AWS Steps plugin or AWS CLI in pipeline scripts.