# Audit Images from ECR

This guide explains how to configure OpenID Connect (OIDC) authentication to download Kloudfuse container images from AWS ECR and verify their signatures in CI/CD pipelines without managing long-lived AWS credentials.

## Overview

When downloading Kloudfuse container images in CI/CD pipelines, you need AWS ECR access to pull images and AWS KMS access to verify their signatures. Instead of storing AWS credentials (access keys) in your CI/CD environment, you can use OIDC to establish a trust relationship between your CI/CD provider (like GitLab) and AWS.

### Benefits of OIDC Authentication

- **No long-lived credentials** - No AWS access keys to manage or rotate
- **Automatic token generation** - CI/CD provider generates short-lived tokens automatically
- **Fine-grained access control** - Limit access to specific projects, branches, or jobs
- **Enhanced security** - Tokens expire automatically and cannot be reused
- **Audit trail** - AWS CloudTrail logs show which jobs assumed roles

## Architecture

The OIDC authentication flow works as follows:

## Setup Process

Setting up OIDC authentication requires configuring both AWS and your CI/CD provider.

### Step 1: Create OIDC Provider in AWS

First, establish trust between AWS and your CI/CD platform’s OIDC provider.

#### For GitLab (Self-Hosted or GitLab.com)

1. Navigate to **AWS IAM Console** → **Identity Providers** → **Add Provider**
2. Configure the OIDC provider:
   - **Provider Type**: OpenID Connect
   - **Provider URL**: Your GitLab OIDC endpoint
   - **Audience**: Your GitLab instance URL

**Example for GitLab Corporate Instance:**
- Provider URL: `https://gitlab-oidc-config.s3.us-west-2.amazonaws.com`
- Audience: `https://gitlab.example.com`

**Example for GitLab.com:**
- Provider URL: `https://gitlab.com`
- Audience: `https://gitlab.com`

### Step 2: Create and Attach IAM Policies for ECR and KMS Access

Create and attach policies that grant permissions to pull Kloudfuse container images from ECR and verify their signatures.

**For ECR Access:**

Choose **one** of the following AWS managed ECR policies based on your needs:
- **AmazonEC2ContainerRegistryReadOnly** (recommended)

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:DescribeImages",
                "ecr:BatchGetImage",
                "ecr:GetLifecyclePolicy",
                "ecr:GetLifecyclePolicyPreview",
                "ecr:ListTagsForResource",
                "ecr:DescribeImageScanFindings"
            ],
            "Resource": "*"
        }
    ]
}
```

- **AmazonEC2ContainerRegistryPullOnly** (minimum privilege)

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken",
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchImportUpstreamImage"
            ],
            "Resource": "*"
        }
    ]
}
```

**For KMS Access:**

Create a custom policy for KMS signature verification:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "KMSSignatureVerification",
            "Effect": "Allow",
            "Action": [
                "kms:GetPublicKey",
                "kms:Verify",
                "kms:DescribeKey"
            ],
            "Resource": [
                "arn:aws:kms:us-west-2:502496443919:alias/kfuse-cosign-signing-key"
            ]
        }
    ]
}
```

**To attach these policies:**
1. Navigate to **AWS IAM Console** → **Policies** → **Create Policy**
2. Create the KMS policy above and name it: `KloudfuseImageSignatureVerifyPolicy`
3. When creating the role in Step 3, attach **both policies**:
   - `AmazonEC2ContainerRegistryReadOnly` **or** `AmazonEC2ContainerRegistryPullOnly` (AWS managed — choose one)
   - `KloudfuseImageSignatureVerifyPolicy` (your custom policy)

### Step 3: Create IAM Role with OIDC Trust Relationship

Create an IAM role that can be assumed by your CI/CD jobs using OIDC tokens.

1. Navigate to **AWS IAM Console** → **Roles** → **Create Role**
2. Select **Web Identity** as the trusted entity type
3. Configure the trust relationship:

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::YOUR_AWS_ACCOUNT_ID:oidc-provider/gitlab.example.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "gitlab.example.com:aud": "https://gitlab.example.com"
                },
                "StringLike": {
                    "gitlab.example.com:sub": "project_path:your-organization/*"
                }
            }
        }
    ]
}
```

1. **Attach the policies** created in Step 2 (`AmazonEC2ContainerRegistryReadOnly` or `AmazonEC2ContainerRegistryPullOnly`, and `KloudfuseImageSignatureVerifyPolicy`)
2. Name the role: `KloudfuseImagePullRole`
3. Note the **Role ARN** - you will need this in your CI/CD pipeline

### Step 4: Configure CI/CD Pipeline

Configure your CI/CD pipeline to use the OIDC role for AWS authentication.

#### GitLab CI/CD Example

```yaml
verify-kfuse-image:
  stage: verify
  image: docker:latest
  services:
    - docker:dind

variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_TLS_CERTDIR: ""
    IMAGE_NAME: ui
    IMAGE_TAG: "0.1.0-b2274ac0"
    COSIGN_VERSION: "v3.0.3"
    KMS_KEY: "awskms:///arn:aws:kms:us-west-2:502496443919:alias/kfuse-cosign-signing-key"
    AWS_ROLE_ARN: "arn:aws:iam::YOUR_AWS_ACCOUNT_ID:role/KloudfuseImageVerificationRole"
    AWS_DEFAULT_REGION: "us-west-2"

id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.example.com

before_script:
    - apk add --no-cache curl aws-cli jq
    - curl -sSfL "https://github.com/sigstore/cosign/releases/download/${COSIGN_VERSION}/cosign-linux-amd64" -o /usr/local/bin/cosign
    - chmod +x /usr/local/bin/cosign
    - |
      echo "Assuming AWS role using OIDC..."

STS_RESPONSE=$(aws sts assume-role-with-web-identity \
        --role-arn "${AWS_ROLE_ARN}" \
        --role-session-name "gitlab-ci-${CI_PROJECT_NAME}-${CI_PIPELINE_ID}" \
        --web-identity-token "${GITLAB_OIDC_TOKEN}" \
        --duration-seconds 3600)

export AWS_ACCESS_KEY_ID=$(echo "${STS_RESPONSE}" | jq -r '.Credentials.AccessKeyId')
      export AWS_SECRET_ACCESS_KEY=$(echo "${STS_RESPONSE}" | jq -r '.Credentials.SecretAccessKey')
      export AWS_SESSION_TOKEN=$(echo "${STS_RESPONSE}" | jq -r '.Credentials.SessionToken')

echo "✅ Successfully authenticated to AWS"

script:
    - |
      IMAGE_REF="502496443919.dkr.ecr.us-west-2.amazonaws.com/kfuse/${IMAGE_NAME}:${IMAGE_TAG}"
      echo "Logging in to Kloudfuse ECR..."
      aws ecr get-login-password --region us-west-2 | \
        docker login --username AWS --password-stdin 502496443919.dkr.ecr.us-west-2.amazonaws.com

echo "Pulling image ${IMAGE_REF}..."
      docker pull "${IMAGE_REF}"

echo "Verifying signature for ${IMAGE_REF}..."
      if cosign verify --key "${KMS_KEY}" "${IMAGE_REF}"; then
        echo "✅ Image signature verified successfully"
      else
        echo "⚠️  Warning: Image signature verification FAILED"
      fi

echo "✅ Image pulled and verified successfully"
```

## Verification and Testing

### Test the Pipeline

1. **Commit the pipeline configuration** to your GitLab repository
2. **Run the pipeline** manually or via a commit
3. **Check the job logs** for:

```text
Assuming AWS role using OIDC...
✅ Successfully authenticated to AWS
Logging in to Kloudfuse ECR...
Login Succeeded
Pulling image 502496443919.dkr.ecr.us-west-2.amazonaws.com/kfuse/ui:0.1.0-b2274ac0...
0.1.0-b2274ac0: Pulling from kfuse/ui
✅ Image pulled and verified successfully
```

### Verify in AWS CloudTrail

Check AWS CloudTrail logs to confirm OIDC authentication:
1. Navigate to **AWS CloudTrail Console** → **Event History**
2. Filter by:
   - **Event name**: `AssumeRoleWithWebIdentity`
   - **User name**: Your role name
3. Inspect the event details to see:
   - Which GitLab project assumed the role
   - Session name (includes pipeline ID)
   - Timestamp and source IP

## Troubleshooting

- **"Not authorized to perform sts:AssumeRoleWithWebIdentity"**
  - *Cause:* OIDC provider not configured or trust relationship incorrect
  - *Solution:* Verify the OIDC provider ARN in the role’s trust relationship matches the provider created in Step 1

- **"Access Denied" with audience mismatch**
  - *Cause:* The `aud` claim in the OIDC token doesn’t match the trust relationship
  - *Solution:* Ensure `id_tokens.GITLAB_OIDC_TOKEN.aud` in your pipeline matches the audience in the trust relationship condition

- **"Access denied" with subject mismatch**
  - *Cause:* The `sub` claim in the OIDC token doesn’t match the trust relationship condition
  - *Solution:* Verify the `StringLike` condition in the trust relationship matches your GitLab project path

- **"Token has expired"**
  - *Cause:* OIDC token lifetime exceeded
  - *Solution:* OIDC tokens are typically valid for 5-10 minutes. Ensure `assume-role-with-web-identity` runs early in the job

- **"Access Denied" when calling KMS operations**
  - *Cause:* IAM role doesn’t have KMS permissions
  - *Solution:* Verify the KMS policy from Step 2 is attached to the role

### Debug OIDC Token Claims

To inspect what claims GitLab is sending in the OIDC token:

```bash
# Add this to your pipeline before_script to decode the token
echo "${GITLAB_OIDC_TOKEN}" | cut -d. -f2 | base64 -d 2>/dev/null | jq .
```

This will show claims like:

```json
{
  "aud": "https://gitlab.example.com",
  "sub": "project_path:your-organization/your-project:ref:refs/heads/main",
  "iss": "https://gitlab.example.com",
  "exp": 1735123456,
  "iat": 1735123156,
  "jti": "abc123..."
}
```

## Security Best Practices

1. **Minimize trust scope** - Use specific `StringLike` conditions to limit which projects/branches can assume the role
2. **Short session duration** - Use the minimum `--duration-seconds` needed for your job (typically 3600 = 1 hour)
3. **Least privilege** - Grant only the KMS permissions needed (`GetPublicKey`, `Verify`, `DescribeKey`)
4. **Monitor CloudTrail** - Set up alerts for unexpected `AssumeRoleWithWebIdentity` events
5. **Review OIDC providers** - Periodically audit and update OIDC provider configurations
6. **Use session names** - Include pipeline/job identifiers in session names for better audit trails

## Advanced Configurations

### Multiple AWS Accounts (Development, Staging, Production)

Use different IAM roles per environment:

```yaml
.verify-template:
  stage: verify
  # ... (common configuration)
  script:
    - |
      case "${CI_ENVIRONMENT_NAME}" in
        production)
          AWS_ROLE_ARN="arn:aws:iam::111111111111:role/KfuseImageVerify-Prod"
          ;;
        staging)
          AWS_ROLE_ARN="arn:aws:iam::222222222222:role/KfuseImageVerify-Staging"
          ;;
        development)
          AWS_ROLE_ARN="arn:aws:iam::333333333333:role/KfuseImageVerify-Dev"
          ;;
      esac

# ... (assume role and verify)

verify-prod:
  extends: .verify-template
  environment: production
  only:
    - main

verify-staging:
  extends: .verify-template
  environment: staging
  only:
    - staging
```

### Cross-Account KMS Access

If your AWS account differs from Kloudfuse’s KMS account (502496443919), you need to:
1. Ensure the KMS key policy allows your account
2. Add `kms:*` permissions for the key ARN in your IAM policy

### Other CI/CD Providers

**GitHub Actions:**

```yaml
- name: Configure AWS credentials
  uses: aws-actions/configure-aws-credentials@v4
  with:
    role-to-assume: arn:aws:iam::YOUR_ACCOUNT:role/KloudfuseImageVerificationRole
    aws-region: us-west-2

- name: Verify image signature
  run: |
    cosign verify \
      --key awskms:///arn:aws:kms:us-west-2:502496443919:alias/kfuse-cosign-signing-key \
      502496443919.dkr.ecr.us-west-2.amazonaws.com/kfuse/ui:0.1.0-b2274ac0
```

**CircleCI:** Use CircleCI’s OIDC token integration with AWS (see [CircleCI OIDC docs](https://circleci.com/docs/openid-connect-tokens/))

**Jenkins:** Configure OIDC authentication using the AWS Steps plugin or AWS CLI in pipeline scripts.