Skip to main content
Version: Dev

Backup and Recovery

An On-Premises AISIX Cloud deployment is recoverable only when its PostgreSQL data and the secrets and configuration that interpret that data are preserved together. A database backup without the matching master keys leaves encrypted credentials unreadable. Configuration files without the matching database cannot reconstruct organizations, resources, or the certificate authority trusted by existing gateways.

Treat the database, encryption keys, deployment configuration, and software version as one recovery set. Keep a matching copy of each asset, protect the set as a secret, and rehearse restoration before relying on it for an upgrade rollback or disaster recovery.

Choose the Recovery Scope

GoalRequired protection
Recover the complete On-Premises control planeBack up the complete PostgreSQL state together with the matching secrets, deployment configuration, and software version.
Roll back an upgradeTake a recovery set immediately before the upgrade. Restore the database before starting the previous control-plane version.
Move or reproduce one organizationUse Organization Backup and Restore. An organization bundle does not replace a complete deployment backup.
Keep gateways serving during a temporary control-plane outageConfigure Offline Resilience. A gateway snapshot cache is a continuity mechanism, not a control-plane backup.

Protect the Recovery Set

Build the recovery set from the assets below, then store and retain it outside the deployment's failure domain.

PostgreSQL State

The control-plane database contains organizations, environments, users, models, policies, encrypted credentials, projected gateway configuration, audit and usage data, and the root certificate authority and gateway-certificate ledgers. Back up the entire database rather than selecting individual application tables.

Use a PostgreSQL-aware backup or a storage snapshot coordinated with PostgreSQL. Do not copy a live database volume as ordinary files. The backup must preserve both the public and auth schemas and the Kine table created by dp-manager.

For an external database, include the connection role, ownership, and required privileges in the database recovery plan. A per-database dump does not preserve cluster-global roles, even when it contains grants that reference cp_api_app. Choose one recovery strategy:

  • preserve cp_api_app with NOSUPERUSER NOBYPASSRLS NOINHERIT NOLOGIN and restore the role-dependent ownership and privilege statements;
  • omit those statements and let cp-api recreate the role and grants at startup.

See External Database for both role contracts. AISIX does not provide a database-backup command or replace the backup tooling and retention policy operated by your database platform.

Encryption and Authentication Secrets

The active master key encrypts stored provider and integration credentials and the root certificate authority's private key. Preserve the active key, its key ID, and every retired key that may still wrap a stored value. Losing required master-key material is not recoverable from the database.

Also preserve the dashboard authentication secret and database credentials. Keep these values in a secret manager or encrypted backup with access controls at least as strict as the database backup.

Deployment Configuration

Preserve the configuration that locates and exposes the restored services. This includes the public control-plane and dp-manager URLs, database connection, private CA mounts, image overrides, and network-access settings. Also preserve any extra environment variables or Kubernetes Secrets supplied outside the default package or chart.

Restoring a database with configuration from another deployment can produce authentication failures, unreadable ciphertext, or a certificate authority that does not match the gateway certificate records.

Release Identity

Record the control-plane package or chart version and the effective image references at the time of backup. Restore that version first. Starting a newer version can migrate the restored database before you have verified the recovery, while starting an older version against a newer schema is unsupported.

Keep version-pinned packages or a tested way to retrieve them, especially for air-gapped deployments. Record intentional image overrides separately because they can outlive normal package or chart upgrades.

Storage, Retention, and Access

A recovery set is a logical match between the database, secrets, configuration, and software version. The artifacts do not have to share one storage location. Separating encryption keys from encrypted database backups can reduce the impact of one storage compromise, provided the recovery inventory identifies the matching copies.

Keep at least one recovery copy outside the deployment's host, cluster, and storage failure domain. Define the recovery point objective, backup frequency, retention period, and deletion controls for each copy. Encrypt backups in transit and at rest, restrict access, and record checksums or another integrity signal with the recovery-set manifest.

Periodically verify that retained package or chart versions and every external secret source are still available. A database backup that outlives its master keys or matching software is not recoverable.

Back Up a Docker Compose Deployment

The packaged deployment stores PostgreSQL data in the aisix-self-hosted_pgdata volume and keeps deployment secrets and settings in ./aisix-self-hosted/.env. The data volume survives ./run.sh down and package extraction, but .env is not included in later package archives and cannot be reconstructed from the volume.

Preserve these assets together:

AssetWhy it is required
Complete PostgreSQL backup or consistent aisix-self-hosted_pgdata snapshotRestores all control-plane and certificate state.
.envPreserves the database password, master key and key ID, dashboard authentication secret, URLs, release pin, and image or runtime overrides.
Matching package or version-pinned package locationRestores the scripts, Compose definition, and default image set for the database schema you backed up.
External files and local deployment customizationRestores private CA certificates and other operator-managed inputs that are not stored in PostgreSQL.

For a cold volume snapshot, stop the stack so PostgreSQL is not writing, snapshot the complete volume, and then restart the same installation. For an online backup, use PostgreSQL-aware tooling that provides a consistent restore point. Verify the backup can be read before returning the deployment to normal change activity.

Do not treat the package archive as a secret backup. Extracting another package preserves an existing .env; it does not recreate a lost one.

Back Up a Helm Deployment

For Helm installations, preserve the same recovery set through the systems that own each asset:

AssetWhere to protect it
Built-in PostgreSQLBack up the PostgreSQL PVC with a database-aware procedure or a coordinated volume snapshot.
External PostgreSQLUse the database service's backup, point-in-time recovery, and role-management procedures.
Master key and dashboard authentication secretPreserve the Secret rendered by the chart and the protected source from which its values were supplied.
Database and optional serving-role credentialsPreserve the configured existing Secrets or the protected values source used to render them.
Effective Helm values and release historyRetain the values source, chart version, namespace, release name, image tags, and intentional overrides.
Additional mounted Secrets, ConfigMaps, and CA filesBack them up through their owning secret or configuration system.

The chart marks its primary application Secret with helm.sh/resource-policy: keep, so a normal Helm uninstall does not remove it. That annotation does not protect the Secret from namespace or cluster loss. Back up its source or an encrypted copy independently.

Treat exported Helm values as sensitive because they may contain database passwords and application secrets. When values reference existing Secrets, backing up the values alone does not preserve the Secret contents.

Take a Change-Scoped Backup

Before an upgrade or another schema-affecting change:

  1. Identify the current control-plane version and effective image overrides.
  2. Capture a consistent PostgreSQL backup.
  3. Capture the matching Compose .env or Helm values and secret sources.
  4. Record the backup time, database identity, deployment name, software version, storage location, and integrity checksum as one recovery-set manifest.
  5. Verify that the backup artifacts are readable and that the required credentials and version-pinned software remain available.

Do not make unrelated configuration or key-rotation changes between these captures. If the database or its encryption-key set changes, take a new recovery set.

Restore the Control Plane

Restore into an isolated target first when possible. Prevent the recovered control plane from sending notifications, reaching production providers, or accepting production gateway connections until you have verified the result.

  1. Select one complete recovery set and obtain its matching package or chart version.
  2. Restore the deployment configuration and secret sources, including all master keys needed by the database.
  3. Restore PostgreSQL while the control-plane services are stopped. For an external database, recreate the documented connection role and privileges. If the backup includes grants that reference cp_api_app, recreate that internal role with the attributes above before restoring the database. If the backup omits role-dependent ownership and privilege statements, let cp-api recreate the internal role and grants when it starts.
  4. Start the matching control-plane version through its normal Compose or Helm deployment path.
  5. Review startup logs and confirm that cp-api, dp-manager, and the dashboard become healthy without database, master-key, migration, or ca_root errors.
  6. Sign in and verify organizations, environments, resources, and gateway registrations.
  7. Confirm that gateways can authenticate to dp-manager and that resource projection reaches the expected revision.
  8. When a safe test provider is reachable, send a controlled request through a non-production gateway and confirm the response and configured telemetry.
  9. Re-enable external integrations and production gateway connectivity only after these checks pass.

Restore the database before starting a rollback version. The AISIX_ALLOW_DOWNGRADE=1 package override acknowledges an intentional rollback; it does not make an incompatible or already-migrated database safe. Follow Upgrade AISIX for the installation-specific rollback procedure.

If the database has gateway certificate rows but no ca_root, prefer restoring a complete database backup. The fresh-bootstrap override creates a new authority and forces every gateway to re-enroll; it does not recover the missing key. See Recover a Missing CA Root.

Rehearse Recovery

A backup is not proven until it has been restored. Rehearse in an isolated environment on a schedule that matches the deployment's risk and change rate.

During a rehearsal:

  • restore the database, secrets, configuration, and matching software from the recorded recovery set;
  • verify that encrypted resources load without master-key errors;
  • confirm that the certificate authority and gateway-certificate records are present;
  • sign in and inspect representative organization and environment resources;
  • connect a non-production gateway and verify resource projection;
  • send a controlled request and verify the response and configured telemetry;
  • record restore time, required manual actions, and any missing external dependency.

After the rehearsal, correct the backup process rather than documenting an unrepeatable workaround. Take a new recovery set whenever the process or required asset list changes.

What Does Not Replace a Deployment Backup

  • A gateway snapshot cache preserves the last accepted gateway configuration for temporary control-plane loss. It cannot reconstruct the control plane.
  • A redacted organization bundle intentionally removes live credentials and certificate-authority state.
  • A full organization bundle covers one organization and applies organization-import rules, including billing reset. It is portable but not a complete control-plane backup.
  • A package archive contains software and deployment templates, not the generated .env or PostgreSQL data.
  • Helm release metadata does not replace the database or externally managed Secret contents.