# AWS Terraform Integration with Kloudfuse

## Pre-Requisites
Enable these artifacts to configure S3 as a Terraform backend:

### S3 bucket
Create an S3 bucket with public access turned off, and remaining defaults enabled.

### Dynamodb table
Create a `dynamodb` table with partition `key = LockID`.

For details, see [Terraform Configuration](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/terraform/#backend).

### Install Terraform
Install the latest Terraform release, and export `terraform` to your executable `PATH`.

### Terraform user
Create an AWS user, or use a role that has enough permissions to access the S3 bucket and provision different resources on AWS.

1. Ensure to add the profile to your AWS credentials.
2. Export the AWS profile before running Terraform.

## Terraform Configuration
You must change the `backend.tf` file in each component folder, according to the created resources in S3 and `dynamodb`. Note that the state key is unique for each component; for a VPC, it can be `vpc.tfstate`.

Example Terraform Configuration

```yaml
tf_version = "~> 1.0"
terraform {
  backend "s3" {
    bucket                 = "bucket-name"
    region                 = "us-west-2"
    key                    = "<resource>.tfstate"
    dynamodb_table         = "dynamodb-table-name"
  }
}
```

## Terraform Variables
You must modify the variables in `variables.tf` for every deployed AWS component.

### vpc_cidr
The IPv4 CIDR block for the VPC. CIDR can be explicitly set, or it can be derived from IPAM using `ipv4_netmask_length` and `ipv4_ipam_pool_id`.
Example:
`"172.159.0.0/16"`

### region
Region where the resource is created.
Example:
`"us-west-2"`

### azs
List of availability zone names or IDs in the region.
Example:
`["us-west-2a", "us-west-2b", "us-west-2c"]`

### public_subnets
A list of public subnets in the region.
Example:
`["172.159.0.0/19", "172.159.32.0/19", "172.159.64.0/19"]`

### domain_name
Domain name for the ACM certificate.
Example:
`"terraform-dev.kloudfuse.io"`

### validation_method
Validation method for ACM certificate.
Example:
`"DNS"`

### cluster_name
EKS cluster name.
Example:
`"kfuse-devops-eks"`

### bucket_deepstore
S3 bucket for the deepstore.
Example:
`"kfuse-test-deepstore-s3"`

### principal_arn_console
The ARN of the IAM principal that requires access to the EKS cluster console to view the nodes attached to the cluster.
Example:
`"arn:aws:iam::783739827:user/terraform"`

### principal_arn_cli
The ARN of the IAM principal that requires access to the EKS cluster resources through the CLI.
Example:
`"arn:aws:iam::783739827:user/terraform"`

### ami_type
AMI type used.
Example:
`"AL2_x86_64"`

### instance_types
Instance type of EKS cluster.
Example:
`["r6i.8xlarge"]`

## Configure Infrastructure for Kloudfuse
Choose the method for using Terraform to configure the infrastructure:
- [Configuration for AWS Accounts without Infrastructure](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/terraform/#config-without-infrastructure)
- [Configuration for AWS Accounts with non-EKS VPC Infrastructure](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/terraform/#config-with-non-eks-vpc)
- [Configuration for AWS Accounts with EKS Cluster](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/terraform/#config-with-esk-cluster)

### Configuration for AWS Accounts without Infrastructure
1. **Provision VPC**: Create the VPC using the terraform files for Kloudfuse, from the `kloudfuse/terraform/networking/` directory. Be sure to modify the variables in the `variables.tf` file to reflect the instance requirements for VPC CIDR, Public subnet CIDR, VPC name, etc.

2. **Provision S3**: Ensure that the Kloudfuse datastore Pinot has access to an S3 bucket for long-term storage. See Terraform resources in the `terraform/s3/` directory.

3. **Provision EKS**: Use the Terraform resources in the `/terraform/eks/` directory to provision the EKS cluster. This will require using `terraform_remote_state` to retrieve VPC S3 Backend values for public subnet IDs.

4. **Provision Route53**: Create a public hosted zone for the ACM certificate using the Terraform files in `terraform/route53/` directory.

5. **Provision ACM**: Create a certificate to enable SSL/TLS using the Terraform files in the `terraform/acm/` directory.

### Configuration for AWS Accounts with non-EKS VPC Infrastructure
1. **Provision S3**: Ensure the Kloudfuse datastore Pinot has access to S3 bucket for long-term storage. See Terraform resources in `terraform/s3/` directory.

2. **Provision EKS**: Use Terraform resources in `/terraform/eks/` directory to provision EKS cluster. Modify variables in `variables.tf` accordingly. Ensure to add proper ARN for cluster users.

3. **Provision Route53**: Create a public hosted zone for ACM certificate using Terraform files in `terraform/route53/` directory.

4. **Provision ACM**: Create certificate to enable SSL/TLS using Terraform files in `terraform/acm/` directory.

### Configuration for AWS Accounts with EKS Cluster
You can deploy Kloudfuse using these options:
- [Terraform and Helm](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/terraform/#terraform-helm)
- [Kubectl Commands and Helm](https://docs.kloudfuse.com/platform/4.1.0/data-collection/cloud-services/aws/terraform/#kubectl-helm)

#### Terraform and Helm
1. Copy `token.json` and your `~/.kube/config` (renamed as `kubeconfig` in `destination`) files to the `/prekfuse` folder.

2. Update the config in `kubeconfig` for the EKS Cluster.

3. Run the Kloudfuse prerequisite Terraform files for setting up the cluster using files in the `terraform/prekfuse/` directory.

4. Login to the Kloudfuse Helm registry.

5. Create a secret for services to pull their Helm charts.

6. Run the Helm upgrade command:

```console
helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
     -n kfuse \
     --version <VERSION> \ (1)
     -f custom-values.yaml
```

#### Kubectl Commands and Helm
1. Login to Kloudfuse Helm registry using `token.json`.

2. Create a secret for Helm:

```console
kubectl create ns kfuse
kubectl config set-context --current --namespace=kfuse
kubectl create secret docker-registry kfuse-image-pull-credentials \
           --namespace='kfuse' --docker-server 'us.gcr.io' --docker-username _json_key \
           --docker-email 'container-registry@mvp-demo-301906.iam.gserviceaccount.com' \
           --docker-password='"$(cat token.json)"'
```

3. Run the following command to install the Kloudfuse chart:

```console
helm upgrade \
   --install -n kfuse kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse \
   --version <VERSION.NUM.BER>  -f custom_values.yaml (1)
```