Skip to main content
API7.ai Logo
Version: Dev

Upgrade an AISIX Gateway on Kubernetes

An AISIX gateway installed with the api7/aisix Helm chart is upgraded as its own release. This procedure applies to AISIX OSS and to gateways connected to AISIX Enterprise or AISIX Cloud. Before the rollout, an open-source AISIX gateway validates its declarative resources with the target image. A connected gateway skips that validation because it receives configuration from its control plane. A gateway on OKD also requires the platform checks in OKD.

For a connected gateway, the product determines where the upgrade procedure starts. For AISIX Enterprise, first follow Upgrade AISIX Enterprise and return here after the upgraded control plane passes verification. API7 operates the AISIX Cloud control plane, so its gateways start directly with this procedure.

Prerequisites​

Before starting the upgrade:

  • Configure kubectl to access the cluster and namespace that contain AISIX.
  • Install Helm 3.15 or a later Helm 3 release, cURL, and jq. For AISIX OSS, also install Docker to validate its resources.
  • Read the release notes for every release after the installed version. Apply any required configuration or observability changes in order.
  • Confirm that the load balancer removes pods whose /readyz check fails.
  • For a multi-replica deployment, confirm that shared Redis is configured when rate-limit counters or cache entries must remain consistent across replicas. If you plan to increase a single-replica deployment before the upgrade, configure shared Redis first or explicitly accept the temporary per-replica behavior. See Prepare AISIX Gateways for Production.

The examples use aisix-dp as the release and namespace, matching the gateway installation guides and dashboard-generated Helm command. If your installation uses another value, run helm list -A, find the release whose CHART column is aisix-<version>, and use its release name and namespace throughout this procedure. A wrong value makes helm upgrade report that the release has no deployed releases and leaves the Deployment lookup below empty.

Set the release details used by the examples:

export GATEWAY_NAMESPACE="aisix-dp"
export GATEWAY_RELEASE="aisix-dp"
export TARGET_VERSION="<target-version>"
export GATEWAY_UPGRADE_TIMEOUT="60m"

The 60-minute timeout is a conservative starting allowance for the chart defaults. These defaults include two replicas, a 1,230-second termination ceiling per pod, and a five-minute startup-probe budget. It is not the expected duration.

Kubernetes can overlap pod termination, and Helm does not require old pods to finish before it considers the updated Deployment ready. Recalculate the timeout when changing replicas, update strategy, termination grace, probe budgets, or when image pulls and scheduling are slower.

Confirm that the cluster can schedule the surge pod; a longer timeout does not resolve a quota or capacity rejection. If an atomic upgrade fails, Helm can use another full timeout period for its rollback, so reserve that time in the maintenance window.

Refresh the API7 chart repository and confirm that the target chart is available:

helm repo add api7 https://charts.api7.ai
helm repo update
helm show chart api7/aisix --version "$TARGET_VERSION"

Capture the Current Release​

Record the installed chart and application versions:

helm list -n "$GATEWAY_NAMESPACE"
kubectl get pods -n "$GATEWAY_NAMESPACE"

Resolve the Deployment name from the Helm release label. This works when the chart uses a name override:

export GATEWAY_DEPLOYMENT="$(kubectl get deployment -n "$GATEWAY_NAMESPACE" \
  -l "app.kubernetes.io/instance=$GATEWAY_RELEASE,app.kubernetes.io/component=gateway" \
  -o jsonpath='{.items[0].metadata.name}')"

printf 'Gateway: %s\n' "$GATEWAY_DEPLOYMENT"

Stop and verify the release name and namespace if the value is empty.

Save only the user-supplied values. The file can contain credentials, so create it with restrictive permissions. Do not export all computed defaults with helm get values --all; carrying old defaults forward can prevent a new chart default from taking effect.

umask 077
touch gateway-values.yaml
chmod 600 gateway-values.yaml
helm get values "$GATEWAY_RELEASE" -n "$GATEWAY_NAMESPACE" -o yaml > gateway-values.yaml

Keep this file with the upgrade records, and remove the local copy after the observation period.

Check whether the values pin an image tag:

helm get values "$GATEWAY_RELEASE" -n "$GATEWAY_NAMESPACE" -o json \
  | jq '{gateway_image_tag: (.image.tag // "")}'

An empty tag uses the target chart's application version. If the value pins an older AISIX image, remove it from the saved values or update it to the target version. Preserve a deliberate custom repository, and use a target-compatible tag for a custom build.

Render the Target Gateway Image​

Render the target Deployment and capture the exact gateway image that Helm will apply. This preserves the chart's registry and any deliberate repository or tag override:

TARGET_GATEWAY_IMAGE="$(
  helm template "$GATEWAY_RELEASE" api7/aisix \
    --namespace "$GATEWAY_NAMESPACE" \
    --version "$TARGET_VERSION" \
    --values gateway-values.yaml \
    --show-only templates/deployment.yaml \
    | awk '
      $1 == "-" && $2 == "name:" && $3 == "aisix" { in_gateway = 1; next }
      in_gateway && $1 == "image:" { gsub(/"/, "", $2); print $2; exit }
    '
)"
export TARGET_GATEWAY_IMAGE

if [ -z "$TARGET_GATEWAY_IMAGE" ]; then
  echo "Target gateway image was not rendered" >&2
else
  printf 'Target gateway image: %s\n' "$TARGET_GATEWAY_IMAGE"
fi

Do not continue if the target gateway image was not rendered.

Validate AISIX OSS Resources​

This section applies only to AISIX OSS. For a gateway connected to AISIX Enterprise or AISIX Cloud, continue to Check the Target Release.

Identify which resource source the release uses without printing the inline resource document:

helm get values "$GATEWAY_RELEASE" -n "$GATEWAY_NAMESPACE" -o json \
  | jq '{inline_resources: (((.standalone.resources // {}) | length) > 0), existing_secret: (.standalone.existingSecret // ""), existing_config_map: (.standalone.existingConfigMap // "")}'

export TARGET_RESOURCES_FILE="$(mktemp "$PWD/aisix-resources.XXXXXX")"
chmod 600 "$TARGET_RESOURCES_FILE"

Use the command for the reported source. For inline resources, render the target chart's managed Secret and extract the complete document:

helm template "$GATEWAY_RELEASE" api7/aisix \
  --namespace "$GATEWAY_NAMESPACE" \
  --version "$TARGET_VERSION" \
  --values gateway-values.yaml \
  --show-only templates/secret.yaml \
  | awk '
    /^  resources\.yaml: \|$/ { in_resources = 1; next }
    in_resources && /^---$/ { exit }
    in_resources { sub(/^    /, ""); print }
  ' > "$TARGET_RESOURCES_FILE"

For an existing Secret or ConfigMap, replace the placeholder with the name reported by the preceding jq command:

# Choose one command.
kubectl get secret -n "$GATEWAY_NAMESPACE" \
  <existing-secret-name> \
  -o jsonpath='{.data.resources\.yaml}' \
  | base64 --decode > "$TARGET_RESOURCES_FILE"

kubectl get configmap -n "$GATEWAY_NAMESPACE" \
  <existing-config-map-name> \
  -o jsonpath='{.data.resources\.yaml}' > "$TARGET_RESOURCES_FILE"

Validate the extracted document with TARGET_GATEWAY_IMAGE, which was captured from the target Deployment render. Export every environment variable referenced by the resource file, and add one -e option for each variable. The example uses OPENAI_API_KEY and CALLER_API_KEY; replace them with the variables in your file.

The command streams the document into the short-lived container instead of bind-mounting the host file. The gateway image can therefore read the document without weakening its host permissions or changing its SELinux label.

docker run --rm -i \
  -e OPENAI_API_KEY \
  -e CALLER_API_KEY \
  --entrypoint /bin/sh "$TARGET_GATEWAY_IMAGE" \
  -c 'umask 077; cat > /tmp/resources.yaml && exec /usr/local/bin/aisix validate --resources /tmp/resources.yaml' \
  < "$TARGET_RESOURCES_FILE" && {
    rm -f "$TARGET_RESOURCES_FILE" &&
    unset TARGET_RESOURCES_FILE
  }

Resolve every validation error before continuing. A failed validation retains the extracted file for inspection and returns a nonzero status. If the resource file does not reference environment variables, omit the -e options.

Check the Target Release​

Have Helm render the upgrade against the live cluster before changing the release. --dry-run=server permits cluster lookups and OpenAPI validation, while --hide-secret prevents Helm from printing rendered Secrets to the terminal or CI log:

helm upgrade "$GATEWAY_RELEASE" api7/aisix \
  --namespace "$GATEWAY_NAMESPACE" \
  --version "$TARGET_VERSION" \
  --values gateway-values.yaml \
  --dry-run=server \
  --hide-secret

This command does not submit the rendered objects to admission. Resolve every reported error before continuing.

Upgrade the Gateway​

Before running Helm, open a second terminal and watch the gateway pods. Leave the watch running until Helm returns:

kubectl get pods -n "$GATEWAY_NAMESPACE" \
  -l "app.kubernetes.io/instance=$GATEWAY_RELEASE,app.kubernetes.io/component=gateway" \
  --watch

If a new pod stalls or restarts, press Ctrl+C to stop the watch, or open a third terminal. Capture the pod's description and logs while Helm is still waiting in the original terminal. An atomic rollback can remove the failed target pods before the Helm command returns; see Troubleshoot a Gateway That Is Not Ready.

Run the upgrade in the original terminal:

helm upgrade "$GATEWAY_RELEASE" api7/aisix \
  --namespace "$GATEWAY_NAMESPACE" \
  --version "$TARGET_VERSION" \
  --values gateway-values.yaml \
  --atomic \
  --timeout "$GATEWAY_UPGRADE_TIMEOUT"

--atomic blocks until the target release becomes ready or Helm finishes rolling it back after a failure. If the command exits with a nonzero status, do not treat a later successful kubectl rollout status as proof that the target version succeeded; it may be reporting the restored revision. Retain the diagnostics captured while Helm was running, then inspect the release history and verify the restored gateway release:

helm history "$GATEWAY_RELEASE" -n "$GATEWAY_NAMESPACE"

With the default rolling-update strategy and readiness probe, a new gateway stays out of Service endpoints until it loads usable configuration. Keep readinessProbe.enabled set to true, and do not select the Recreate strategy when the upgrade must avoid traffic interruption. A single-replica deployment can still have an interruption while its only pod is replaced.

If the Helm command succeeds, confirm that the target rollout is complete and every gateway pod is ready:

kubectl rollout status -n "$GATEWAY_NAMESPACE" \
  "deployment/$GATEWAY_DEPLOYMENT" \
  --timeout="$GATEWAY_UPGRADE_TIMEOUT"

kubectl get pods -n "$GATEWAY_NAMESPACE"

Verify the Upgrade​

Set AISIX_PROXY to the gateway origin without a trailing slash, then list the models available to the caller:

export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="<caller-api-key>"

curl -fsS "$AISIX_PROXY/v1/models" \
  -H "Authorization: Bearer $AISIX_API_KEY"

Send a known-good inference request for one of the listed models. Check the access logs, metrics, and other telemetry that the deployment depends on. Keep the saved values until the gateway has operated normally through the observation period defined by your change policy.

For a gateway connected to AISIX Enterprise or AISIX Cloud, also confirm in the dashboard that every upgraded gateway reports the target version and a healthy status.

Roll Back​

If Helm's atomic upgrade fails, it restores the previous release automatically. Diagnose the failure before retrying.

To roll back after a successful upgrade, identify the previous revision and restore it:

helm history "$GATEWAY_RELEASE" -n "$GATEWAY_NAMESPACE"

helm rollback "$GATEWAY_RELEASE" <previous-revision> \
  --namespace "$GATEWAY_NAMESPACE" \
  --wait \
  --timeout "$GATEWAY_UPGRADE_TIMEOUT"

Confirm that the restored pods become ready and repeat the traffic verification.

Troubleshoot a Gateway That Is Not Ready​

After stopping the watch, run these diagnostics while Helm is still waiting, using the new target pod reported by the watch. Alternatively, run them in a third terminal. After a failed atomic upgrade returns, the failed pod may already be gone; use the captured output and events, then confirm the restored revision with the release-history command above.

kubectl get events -n "$GATEWAY_NAMESPACE" --sort-by=.lastTimestamp

kubectl get pods -n "$GATEWAY_NAMESPACE" \
  -l "app.kubernetes.io/instance=$GATEWAY_RELEASE,app.kubernetes.io/component=gateway"

kubectl describe pod -n "$GATEWAY_NAMESPACE" <non-ready-gateway-pod>

kubectl logs -n "$GATEWAY_NAMESPACE" \
  <non-ready-gateway-pod> \
  --all-containers \
  --tail=300

If a container restarted, use kubectl describe pod to identify its name, then read that container's previous logs with kubectl logs --previous.

For AISIX OSS resource-validation errors, confirm that the saved standalone values identify the complete resources file and that the gateway can reach every configured upstream endpoint.

For a gateway connected to AISIX Enterprise or AISIX Cloud, verify that the pod can reach the existing data-plane manager endpoint over mutual TLS. If the endpoint uses a Route, confirm that the Route uses TLS passthrough. In every case, confirm that the endpoint host name still matches the address in the gateway configuration.