Standalone Envoy Gateway

When to Use Standalone Envoy

Use this pattern when any of the following apply:

• You need to fail over between Kloudfuse clusters in different AZs or regions without DNS changes or TLS re-issuance.
• You want to spin up a new Kloudfuse install in parallel with an existing one and cut traffic over once verified.
• The Envoy load balancer must survive helm delete of the Kloudfuse release (for example, blue/green upgrades).

For a single-cluster install where Envoy and Kloudfuse are managed together, use the in-chart configuration described in Configure Envoy Ingress instead.

Prerequisites

• Kubernetes cluster 1.27 or later with kubectl configured.
• Helm 3.x.
• Gateway API CRDs installed cluster-wide (see Install Envoy CRDs).
• cert-manager installed cluster-wide (or AWS ACM configured) for TLS.
• A reserved static external IP (and an internal IP if the deployment needs an internal LB).
• OCI registry access for oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway. The login commands below use the gcloud CLI (authenticated to a principal with read access to the registry); if gcloud is not available, substitute any other method of obtaining a registry token — e.g. helm registry login -u _json_key --password-stdin …​ < key.json with a service account key — before running helm install.
• Image pull secret for us.gcr.io/mvp-demo-301906/kfuse/envoy-gateway-fips in the install namespace — referenced by global.imagePullSecrets in your values.yaml (the controller and proxy won’t start without it). Copy the existing kfuse-image-pull-credentials secret from the Kloudfuse namespace, or create a dockerconfigjson secret from your registry key:

kubectl get secret kfuse-image-pull-credentials -n <kfuse-namespace> -o yaml \
    | grep -v 'namespace:\|resourceVersion:\|uid:\|creationTimestamp:' \
    | kubectl apply -n envoy-gateway-system -f -

Disable the Bundled Envoy in the Kloudfuse Chart

Before installing the standalone controller, configure each Kloudfuse install so it emits routing manifests only — no controller, no proxy pods, no load balancer.

Add the following to each Kloudfuse custom_values.yaml:

envoy-gateway:
  enabled: false                          (1)
  installGatewayRoutes: true              (2)
  gatewayClassName: "envoy-gateway-az1"   (3)
  envoyService:
    internal:
      enabled: true                       (4)
      gatewayClassName: "envoy-gateway-az1-internal"

ingress-nginx:
  enabled: false
  installIngressRules: false

Configure the Standalone Controller

Create a values.yaml for the standalone envoy-gateway chart. The release must:

• Pin the controller and EnvoyProxy pods to the target AZ’s node group via nodeSelector and tolerations.

• Set a unique gatewayClassName per AZ so each Kloudfuse install can target the right gateway.

• Configure the cloud load balancer (external + optional internal) to match the IP / EIP / DNS already in use.

• AWS (EKS)

• GCP (GKE)

• Azure (AKS)

global:
  # Image pull credentials for us.gcr.io/mvp-demo-301906/kfuse/envoy-gateway-fips.
  # Required — without this, both the controller and the EnvoyProxy data plane
  # pods will ImagePullBackOff. The named secret must already exist in the
  # install namespace (see Prerequisites for how to copy it).
  imagePullSecrets:
    - kfuse-image-pull-credentials       (1)

# Optional: set true only if your cluster uses kfRoles node pools
  # (kf_role label/taint); otherwise pin via deployment.pod.* below.
  # kfRoles:
  #   enabled: true

# GatewayClass name referenced by Kloudfuse installs' envoy-gateway.gatewayClassName
gatewayClassName: "envoy-gateway-az1"      (2)

# External LB Service shape — must match the NLB / EIPs the Nginx or
# previous Envoy install was using, so DNS records do not change.
envoyService:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
    service.beta.kubernetes.io/aws-load-balancer-eip-allocations: <YOUR_EIP_ALLOC_IDS>  (3)
  patch:
    externalTrafficPolicy: Local
  external:
    enabled: true
  internal:
    enabled: true
    gatewayClassName: "envoy-gateway-az1-internal"   (4)
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
      service.beta.kubernetes.io/aws-load-balancer-type: nlb

# Pin to the AZ's node group
department:
  pod:
    nodeSelector:
      ng_label: <az-name>                  (5)
    tolerations:
    - key: "ng_taint"
      operator: "Equal"
      value: "<az-name>"
      effect: "NoSchedule"

# certgen Job runs the same pod-placement constraints
certgen:
  job:
    nodeSelector:
      ng_label: <az-name>
    tolerations:
    - key: "ng_taint"
      operator: "Equal"
      value: "<az-name>"
      effect: "NoSchedule"

Install the Standalone Controller

Authenticate against the OCI registry, then install the chart into a dedicated namespace:

gcloud auth print-access-token | helm registry login -u oauth2accesstoken \
  --password-stdin us-east1-docker.pkg.dev

helm upgrade --install envoy-gateway \
  oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway \
  --version <VERSION> \
  --namespace envoy-gateway-system \
  --create-namespace \
  -f values.yaml

Repeat this step once per AZ, each with a different <release-name>, a distinct gatewayClassName (envoy-gateway-az1, envoy-gateway-az2, …), and the AZ-specific node selector / load balancer config.

Verify the Install

At install time the chart creates only the controller, the EnvoyProxy CR, and the GatewayClass. The data plane pods and cloud load balancer come up lazily — when a Gateway first references the GatewayClass, which happens when you install Kloudfuse against it. So after helm install, just confirm the control plane:

kubectl get pods -n envoy-gateway-system          # controller Running
kubectl get gatewayclass envoy-gateway-az1        # ACCEPTED: True

Install Kloudfuse Against the Standalone Gateway

With the standalone controller running, install or upgrade each Kloudfuse cluster using the custom_values.yaml from Disable the Bundled Envoy (Envoy off, routes on, gatewayClassName matching the standalone release). Then verify it registered against the gateway:

# Gateway should show ADDRESS = the standalone LB IP, PROGRAMMED: True
kubectl get gateway -n <kfuse-namespace>

# All HTTPRoutes accepted
kubectl get httproute -n <kfuse-namespace>

Migrating from standalone Nginx to Envoy (in-place LB repoint)

If the cluster currently serves traffic through a standalone Nginx controller (its own Helm release, with Kloudfuse emitting only Ingress rules), you can cut over to Envoy in place — reusing the existing Nginx load balancer so the IP / EIPs and DNS never change. It is the same trick the in-chart migration uses ( Upgrade from Nginx to Envoy): repoint the Nginx LB Service selector at the Envoy proxy pods. The difference is that a standalone Nginx LB is owned by the upstream ingress-nginx chart, which has no envoyMigration logic — so the selector swap is done manually.

AZ / Region Failover

Standalone Envoy makes failover an in-place helm upgrade of the standalone release — no DNS change, no TLS re-issuance, no LB recreation.

The typical procedure:

1. Bring up a fresh Kloudfuse install in the target AZ’s namespace, pointing at the new AZ’s GatewayClass (gatewayClassName: "envoy-gateway-az2"). Verify Kloudfuse is healthy.

kubectl get pods -n <kfuse-az2-namespace>
kubectl get gateway,httproute -n <kfuse-az2-namespace>

2. Upgrade the standalone Envoy release so its gatewayClassName matches the new AZ:

# values.yaml for the standalone release
gatewayClassName: "envoy-gateway-az2"

helm upgrade envoy-gateway \
     oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway \
     --version <VERSION> \
     --namespace envoy-gateway-system \
     -f values.yaml

3. Confirm the new AZ’s Gateway is PROGRAMMED: True and traffic flows through:

kubectl get gateway -n <kfuse-az2-namespace>
curl -sk https://<your-dns-host>/ -o /dev/null -w "%{http_code}\n"

4. Drain the old AZ’s Kloudfuse install once you’ve verified the new one is serving traffic.

Uninstall

helm delete envoy-gateway -n envoy-gateway-system removes the controller, but the EnvoyProxy data plane Deployments and Services it created at runtime are not owned by Helm and stay behind. Clean them up explicitly:

kubectl delete deploy,svc -l app.kubernetes.io/managed-by=envoy-gateway \
  -n envoy-gateway-system

for gc in $(kubectl get gatewayclass -o name | grep envoy-gateway); do
  kubectl patch "$gc" --type=merge -p '{"metadata":{"finalizers":[]}}'
  kubectl delete "$gc"
done

kubectl delete namespace envoy-gateway-system

Troubleshooting