# Standalone Envoy Gateway

## When to Use Standalone Envoy

Use this pattern when **any** of the following apply:

- You need to fail over between Kloudfuse clusters in different AZs or regions without DNS changes or TLS re-issuance.
- You want to spin up a new Kloudfuse install in parallel with an existing one and cut traffic over once verified.
- The Envoy load balancer must survive `helm delete` of the Kloudfuse release (for example, blue/green upgrades).

|     |     |
| --- | --- |
|  | Failover **repoints** one standalone release between clusters, keeping a single IP / set of EIPs — it does not run two load balancers on the same address (an IP/EIP attaches to one LB at a time). For true side-by-side, give each LB distinct IPs and switch DNS once verified. |

For a single-cluster install where Envoy and Kloudfuse are managed together, use the in-chart configuration described in [Configure Envoy Ingress](https://docs.kloudfuse.com/platform/4.1.0/setup/network/envoy/) instead.

## Prerequisites

- Kubernetes cluster 1.27 or later with `kubectl` configured.
- Helm 3.x.
- Gateway API CRDs installed cluster-wide (see [Install Envoy CRDs](https://docs.kloudfuse.com/platform/4.1.0/setup/network/envoy/#prereq-crd)).
- cert-manager installed cluster-wide (or AWS ACM configured) for TLS.
- A reserved static external IP (and an internal IP if the deployment needs an internal LB).
- OCI registry access for `oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway`. The login commands below use the `gcloud` CLI (authenticated to a principal with read access to the registry); if `gcloud` is not available, substitute any other method of obtaining a registry token — e.g. `helm registry login -u _json_key --password-stdin …​ < key.json` with a service account key — before running `helm install`.
- Image pull secret for `us.gcr.io/mvp-demo-301906/kfuse/envoy-gateway-fips` in the install namespace — referenced by `global.imagePullSecrets` in your `values.yaml` (the controller and proxy won’t start without it). Copy the existing `kfuse-image-pull-credentials` secret from the Kloudfuse namespace, or create a `dockerconfigjson` secret from your registry key:

```bash
kubectl get secret kfuse-image-pull-credentials -n <kfuse-namespace> -o yaml \
    | grep -v 'namespace:\|resourceVersion:\|uid:\|creationTimestamp:' \
    | kubectl apply -n envoy-gateway-system -f -
```

## Disable the Bundled Envoy in the Kloudfuse Chart

Before installing the standalone controller, configure each Kloudfuse install so it emits routing manifests only — no controller, no proxy pods, no load balancer.

Add the following to each Kloudfuse `custom_values.yaml`:

```yaml
envoy-gateway:
  enabled: false                          (1)
  installGatewayRoutes: true              (2)
  gatewayClassName: "envoy-gateway-az1"   (3)
  envoyService:
    internal:
      enabled: true                       (4)
      gatewayClassName: "envoy-gateway-az1-internal"

ingress-nginx:
  enabled: false
  installIngressRules: false
```

|     |     |
| --- | --- |
| **1** | `enabled: false` skips the controller Deployment, the EnvoyProxy data plane, and the load balancer Service. The standalone release provides all three. |
| **2** | `installGatewayRoutes: true` keeps the `Gateway`, `HTTPRoute`, `SecurityPolicy`, and `BackendTrafficPolicy` resources — these tell the standalone controller where to send traffic for this Kloudfuse install. |
| **3** | Must match the **external**`GatewayClass` name produced by the standalone release. See [Configure the Standalone Controller](https://docs.kloudfuse.com/platform/4.1.0/setup/network/envoy-standalone/#configure-standalone) for naming. |
| **4** | Only if you run an **internal** load balancer. The internal `Gateway` is a **second**`GatewayClass` and must be wired separately: set `envoyService.internal.gatewayClassName` here to match the internal class produced by the standalone release (`envoyService.internal.gatewayClassName` there). Omit this whole `envoyService` block if you have no internal LB. |

## Configure the Standalone Controller

Create a `values.yaml` for the standalone `envoy-gateway` chart. The release must:

- Pin the controller and EnvoyProxy pods to the target AZ’s node group via `nodeSelector` and `tolerations`.
- Set a unique `gatewayClassName` per AZ so each Kloudfuse install can target the right gateway.
- Configure the cloud load balancer (external + optional internal) to match the IP / EIP / DNS already in use.

- AWS (EKS)
- GCP (GKE)
- Azure (AKS)

```yaml
global:
  # Image pull credentials for us.gcr.io/mvp-demo-301906/kfuse/envoy-gateway-fips.
  # Required — without this, both the controller and the EnvoyProxy data plane
  # pods will ImagePullBackOff. The named secret must already exist in the
  # install namespace (see Prerequisites for how to copy it).
  imagePullSecrets:
    - kfuse-image-pull-credentials       (1)

# Optional: set true only if your cluster uses kfRoles node pools
  # (kf_role label/taint); otherwise pin via deployment.pod.* below.
  # kfRoles:
  #   enabled: true

# GatewayClass name referenced by Kloudfuse installs' envoy-gateway.gatewayClassName
gatewayClassName: "envoy-gateway-az1"      (2)

# External LB Service shape — must match the NLB / EIPs the Nginx or
# previous Envoy install was using, so DNS records do not change.
envoyService:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-eip-allocations: <YOUR_EIP_ALLOC_IDS>  (3)
  patch:
    externalTrafficPolicy: Local
  external:
    enabled: true
  internal:
    enabled: true
    gatewayClassName: "envoy-gateway-az1-internal"   (4)
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
      service.beta.kubernetes.io/aws-load-balancer-type: nlb

# Pin to the AZ's node group
department:
  pod:
    nodeSelector:
      ng_label: <az-name>                  (5)
    tolerations:
    - key: "ng_taint"
      operator: "Equal"
      value: "<az-name>"
      effect: "NoSchedule"

# certgen Job runs the same pod-placement constraints
certgen:
  job:
    nodeSelector:
      ng_label: <az-name>
    tolerations:
    - key: "ng_taint"
      operator: "Equal"
      value: "<az-name>"
      effect: "NoSchedule"
```

## Install the Standalone Controller

Authenticate against the OCI registry, then install the chart into a dedicated namespace:

```bash
gcloud auth print-access-token | helm registry login -u oauth2accesstoken \
  --password-stdin us-east1-docker.pkg.dev

helm upgrade --install envoy-gateway \
  oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway \
  --version <VERSION> \
  --namespace envoy-gateway-system \
  --create-namespace \
  -f values.yaml
```

Repeat this step **once per AZ**, each with a different `<release-name>`, a distinct `gatewayClassName` (`envoy-gateway-az1`, `envoy-gateway-az2`, …), and the AZ-specific node selector / load balancer config.

### Verify the Install

At install time the chart creates only the controller, the `EnvoyProxy` CR, and the `GatewayClass`. The data plane pods and cloud load balancer come up **lazily** — when a `Gateway` first references the `GatewayClass`, which happens when you install Kloudfuse against it. So after `helm install`, just confirm the control plane:

```bash
kubectl get pods -n envoy-gateway-system          # controller Running
kubectl get gatewayclass envoy-gateway-az1        # ACCEPTED: True
```

## Install Kloudfuse Against the Standalone Gateway

With the standalone controller running, install or upgrade each Kloudfuse cluster using the `custom_values.yaml` from [Disable the Bundled Envoy](https://docs.kloudfuse.com/platform/4.1.0/setup/network/envoy-standalone/#disable-bundled-envoy) (Envoy off, routes on, `gatewayClassName` matching the standalone release). Then verify it registered against the gateway:

```bash
# Gateway should show ADDRESS = the standalone LB IP, PROGRAMMED: True
kubectl get gateway -n <kfuse-namespace>

# All HTTPRoutes accepted
kubectl get httproute -n <kfuse-namespace>
```

### Migrating from standalone Nginx to Envoy (in-place LB repoint)

If the cluster currently serves traffic through a **standalone Nginx** controller (its own Helm release, with Kloudfuse emitting only Ingress rules), you can cut over to Envoy **in place** — reusing the existing Nginx load balancer so the IP / EIPs and DNS never change. It is the same trick the in-chart migration uses ( [Upgrade from Nginx to Envoy](https://docs.kloudfuse.com/platform/4.1.0/setup/network/envoy/#migration)): repoint the Nginx LB Service selector at the Envoy proxy pods. The difference is that a standalone Nginx LB is owned by the upstream `ingress-nginx` chart, which has no `envoyMigration` logic — so the selector swap is done manually.

## AZ / Region Failover

Standalone Envoy makes failover an in-place `helm upgrade` of the standalone release — no DNS change, no TLS re-issuance, no LB recreation.

The typical procedure:

1. Bring up a fresh Kloudfuse install in the target AZ’s namespace, pointing at the **new** AZ’s `GatewayClass` (`gatewayClassName: "envoy-gateway-az2"`). Verify Kloudfuse is healthy.

```bash
kubectl get pods -n <kfuse-az2-namespace>
kubectl get gateway,httproute -n <kfuse-az2-namespace>
```

2. Upgrade the standalone Envoy release so its `gatewayClassName` matches the new AZ:

```yaml
# values.yaml for the standalone release
gatewayClassName: "envoy-gateway-az2"
```

```bash
helm upgrade envoy-gateway \
     oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway \
     --version <VERSION> \
     --namespace envoy-gateway-system \
     -f values.yaml
```

3. Confirm the new AZ’s `Gateway` is `PROGRAMMED: True` and traffic flows through:

```bash
kubectl get gateway -n <kfuse-az2-namespace>
curl -sk https://<your-dns-host>/ -o /dev/null -w "%{http_code}\n"
```

4. Drain the old AZ’s Kloudfuse install once you’ve verified the new one is serving traffic.

## Uninstall

`helm delete envoy-gateway -n envoy-gateway-system` removes the controller, but the EnvoyProxy data plane Deployments and Services it created at runtime are **not** owned by Helm and stay behind. Clean them up explicitly:

```bash
kubectl delete deploy,svc -l app.kubernetes.io/managed-by=envoy-gateway \
  -n envoy-gateway-system

for gc in $(kubectl get gatewayclass -o name | grep envoy-gateway); do
  kubectl patch "$gc" --type=merge -p '{"metadata":{"finalizers":[]}}'
  kubectl delete "$gc"
done

kubectl delete namespace envoy-gateway-system
```

## Troubleshooting

| Symptom | Cause / fix |
| --- | --- |
| `Gateway` stays `PROGRAMMED: False` on the Kloudfuse side | The `gatewayClassName` in the Kloudfuse `custom_values.yaml` does not match the `gatewayClassName` set in the standalone release. Check both values and re-run `helm upgrade` on Kloudfuse. |
| `HTTPRoute` objects show `Accepted: False` with `ParentRefNotAllowed` | The standalone Envoy’s `GatewayClass` does not allow the Kloudfuse namespace to attach routes. Confirm the standalone install’s `allowedRoutes` configuration includes the namespaces that contain Kloudfuse `HTTPRoute` objects. |
| LB IP changed after failover | The `loadBalancerIP` (GCP/Azure) or EIP allocations (AWS) in the standalone release’s `values.yaml` must stay constant across upgrades. Confirm the values file used for the failover upgrade still pins the same IP / EIPs. |
| Stale `EnvoyProxy` pods after switching `gatewayClassName` | The controller reconciles within ~30s. If pods persist beyond a minute, check `kubectl logs -n envoy-gateway-system deploy/envoy-gateway`. Stale pods owned by the old `GatewayClass` can be removed with `kubectl delete pod -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gatewayclass=<old-name>`. |
| `helm install` fails with `cannot re-use a name that is still in use` | A standalone release of the same name already exists. List with `helm list -n envoy-gateway-system`; either `helm upgrade` the existing release or pick a new release name.