Standalone Envoy Gateway

Table of Contents

• When to Use Standalone Envoy
• Prerequisites
• Disable the Bundled Envoy in the Kloudfuse Chart
• Configure the Standalone Controller
• Install the Standalone Controller
  • Verify the Install
• Install Kloudfuse Against the Standalone Gateway
  • Migrating from standalone Nginx to Envoy (in-place LB repoint)
• AZ / Region Failover
• Uninstall
• Troubleshooting

In a multi-AZ or multi-region deployment, the Envoy Gateway control plane must outlive any individual Kloudfuse cluster. During a failover, bring up a fresh Kloudfuse install in the target AZ and then repoint the existing Envoy Gateway at the new cluster.

To support that, install envoy-gateway as a separate Helm release in its own namespace, outside the Kloudfuse chart. Each Kloudfuse install then references the shared gateway by GatewayClass name.

When to Use Standalone Envoy

Use this pattern when any of the following apply:

• You need to fail over between Kloudfuse clusters in different AZs or regions without DNS changes or TLS re-issuance.
• You want to spin up a new Kloudfuse install in parallel with an existing one and cut traffic over once verified.
• The Envoy load balancer must survive helm delete of the Kloudfuse release (for example, blue/green upgrades).

For a single-cluster install where Envoy and Kloudfuse are managed together, use the in-chart configuration described in Configure Envoy Ingress instead.

Prerequisites

• Kubernetes cluster 1.27 or later with kubectl configured.
• Helm 3.x.
• Gateway API CRDs installed cluster-wide.
• cert-manager installed cluster-wide (or AWS ACM configured) for TLS.
• A reserved static external IP (and an internal IP if the deployment needs an internal LB).
• OCI registry access for oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway.
• Image pull secret for us.gcr.io/mvp-demo-301906/kfuse/envoy-gateway-fips in the install namespace.

kubectl get secret kfuse-image-pull-credentials -n <kfuse-namespace> -o yaml \
    | grep -v 'namespace:\|resourceVersion:\|uid:\|creationTimestamp:' \
    | kubectl apply -n envoy-gateway-system -f -

Disable the Bundled Envoy in the Kloudfuse Chart

Before installing the standalone controller, configure each Kloudfuse install so it emits routing manifests only.

Add to each Kloudfuse custom_values.yaml:

envoy-gateway:
  enabled: false
  installGatewayRoutes: true
  gatewayClassName: "envoy-gateway-az1"
  envoyService:
    internal:
      enabled: true
      gatewayClassName: "envoy-gateway-az1-internal"

ingress-nginx:
  enabled: false
  installIngressRules: false

Configure the Standalone Controller

Create a values.yaml for the standalone envoy-gateway chart. The release must:

• Pin the controller and EnvoyProxy pods to the target AZ’s node group via nodeSelector and tolerations.
• Set a unique gatewayClassName per AZ.

global:
  imagePullSecrets:
    - kfuse-image-pull-credentials

gatewayClassName: "envoy-gateway-az1"

envoyService:
  external:
    enabled: true
  internal:
    enabled: true
    gatewayClassName: "envoy-gateway-az1-internal"

deployment:
  pod:
    nodeSelector:
      ng_label: <az-name>
    tolerations:
    - key: "ng_taint"
      operator: "Equal"
      value: "<az-name>"
      effect: "NoSchedule"

Install the Standalone Controller

Authenticate against the OCI registry, then install the chart into a dedicated namespace:

gcloud auth print-access-token | helm registry login -u oauth2accesstoken \
  --password-stdin us-east1-docker.pkg.dev

helm upgrade --install envoy-gateway \
  oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/envoy-gateway \
  --version <VERSION> \
  --namespace envoy-gateway-system \
  --create-namespace \
  -f values.yaml

Verify the Install

kubectl get pods -n envoy-gateway-system          # controller Running
kubectl get gatewayclass envoy-gateway-az1        # ACCEPTED: True

Install Kloudfuse Against the Standalone Gateway

With the standalone controller running, install or upgrade each Kloudfuse cluster using the custom_values.yaml from above. Then verify it registered against the gateway:

kubectl get gateway -n <kfuse-namespace>

kubectl get httproute -n <kfuse-namespace>

Migrating from standalone Nginx to Envoy (in-place LB repoint)

If the cluster currently serves traffic through a standalone Nginx controller, reuse the existing Nginx load balancer so the IP / EIPs and DNS never change.

1. Create the standalone Envoy values.yaml.

[...]  # see the migration section for more details.

2. helm upgrade --install the standalone Envoy controller.

3. Edit the Kloudfuse custom_values.yaml.

envoy-gateway:
     enabled: false
     installGatewayRoutes: true
     gatewayClassName: "<standalone-class>"
ingress-nginx:
     enabled: false
     installIngressRules: true

4. helm upgrade Kloudfuse to apply it.

AZ / Region Failover

Standalone Envoy makes failover easier with no DNS change, no TLS re-issuance, and no LB recreation.

1. Bring up a fresh Kloudfuse install in the new AZ.

kubectl get pods -n <kfuse-az2-namespace>

2. Upgrade the standalone Envoy release to match the new AZ:

gatewayClassName: "envoy-gateway-az2"

3. Confirm the new AZ’s Gateway is PROGRAMMED: True:

kubectl get gateway -n <kfuse-az2-namespace>

4. Drain the old AZ’s Kloudfuse install once you’ve verified the new one is serving traffic.

Uninstall

kubectl delete deploy,svc -l app.kubernetes.io/managed-by=envoy-gateway \
  -n envoy-gateway-system

Troubleshooting