a6-recipe-mtls

Overview

Mutual TLS (mTLS) ensures both the client and server verify each other's identity via TLS certificates. Standard TLS only verifies the server; mTLS adds client certificate verification.

With APISIX and the a6 CLI, you can configure:

1. Client → APISIX mTLS: Require clients to present valid certificates
2. APISIX → Upstream mTLS: Present client certificates when connecting to backends
3. End-to-end mTLS: Both directions simultaneously

When to Use

• Zero-trust networking between services
• Secure service-to-service communication in microservices
• Compliance requirements mandating mutual authentication
• Replace or supplement API key authentication with certificate-based auth
• Internal APIs that should only be accessible by authorized services

Concepts

Term	Description
CA certificate	Certificate Authority cert used to verify client/server certs
Server certificate	Presented by APISIX to clients (standard TLS)
Client certificate	Presented by clients to APISIX (mTLS verification)
Upstream TLS	APISIX presents a client cert to the upstream backend

Part 1: Client → APISIX mTLS

Require clients to present a valid TLS certificate when connecting to APISIX.

1. Create SSL resource with CA for client verification

a6 ssl create -f - <<'EOF'
{
  "id": "mtls-domain",
  "cert": "<SERVER_CERTIFICATE_PEM>",
  "key": "<SERVER_PRIVATE_KEY_PEM>",
  "snis": ["api.example.com"],
  "client": {
    "ca": "<CA_CERTIFICATE_PEM>"
  }
}
EOF

Fields:

• cert / key: Server certificate and private key (presented to clients)
• snis: Server Name Indications — domain names this certificate covers
• client.ca: CA certificate used to verify client certificates
• client.depth: (optional) Maximum certificate chain depth for verification

2. Create a route on the protected domain

a6 route create -f - <<'EOF'
{
  "id": "secure-api",
  "uri": "/api/*",
  "host": "api.example.com",
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "backend:8080": 1
    }
  }
}
EOF

3. Test with client certificate

# With valid client cert — succeeds
curl --cert client.crt --key client.key --cacert ca.crt \
  https://api.example.com:9443/api/health

# Without client cert — fails with SSL handshake error
curl --cacert ca.crt https://api.example.com:9443/api/health

Part 2: APISIX → Upstream mTLS

Configure APISIX to present a client certificate when connecting to backends.

1. Create upstream with TLS client certificate

a6 upstream create -f - <<'EOF'
{
  "id": "mtls-backend",
  "type": "roundrobin",
  "scheme": "https",
  "nodes": {
    "secure-backend:443": 1
  },
  "tls": {
    "client_cert": "<CLIENT_CERTIFICATE_PEM>",
    "client_key": "<CLIENT_PRIVATE_KEY_PEM>"
  }
}
EOF

Fields:

• scheme: Must be "https" for TLS connections to upstream
• tls.client_cert: Client certificate APISIX presents to the upstream
• tls.client_key: Private key for the client certificate
• pass_host: Set to "pass" (default) or "rewrite" if upstream expects a specific Host header

2. Create route using this upstream

a6 route create -f - <<'EOF'
{
  "id": "api",
  "uri": "/api/*",
  "upstream_id": "mtls-backend"
}
EOF

Part 3: End-to-End mTLS

Combine both: clients verify themselves to APISIX, and APISIX verifies itself to the upstream.

1. SSL for client → APISIX mTLS

a6 ssl create -f - <<'EOF'
{
  "id": "frontend-mtls",
  "cert": "<APISIX_SERVER_CERT>",
  "key": "<APISIX_SERVER_KEY>",
  "snis": ["api.example.com"],
  "client": {
    "ca": "<CLIENT_CA_CERT>"
  }
}
EOF

2. Upstream for APISIX → backend mTLS

a6 upstream create -f - <<'EOF'
{
  "id": "secure-backend",
  "type": "roundrobin",
  "scheme": "https",
  "nodes": {
    "internal-service:443": 1
  },
  "tls": {
    "client_cert": "<APISIX_CLIENT_CERT>",
    "client_key": "<APISIX_CLIENT_KEY>"
  }
}
EOF

3. Route connecting both

a6 route create -f - <<'EOF'
{
  "id": "e2e-mtls-api",
  "uri": "/api/*",
  "host": "api.example.com",
  "upstream_id": "secure-backend"
}
EOF

Common Patterns

Multiple domains with different CAs

# Domain A: internal services
a6 ssl create -f - <<'EOF'
{
  "id": "internal-mtls",
  "cert": "<INTERNAL_CERT>",
  "key": "<INTERNAL_KEY>",
  "snis": ["internal.example.com"],
  "client": {
    "ca": "<INTERNAL_CA>"
  }
}
EOF

# Domain B: partner services
a6 ssl create -f - <<'EOF'
{
  "id": "partner-mtls",
  "cert": "<PARTNER_CERT>",
  "key": "<PARTNER_KEY>",
  "snis": ["partner.example.com"],
  "client": {
    "ca": "<PARTNER_CA>"
  }
}
EOF

Using APISIX Secret for certificate management

Store certificates in external secret managers (Vault, AWS, etc.):

# Create a secret reference
a6 secret create -f - <<'EOF'
{
  "id": "vault/mtls-certs",
  "uri": "https://vault.example.com/v1/secret/data/mtls"
}
EOF

Certificate rotation

Update certificates without downtime:

a6 ssl update mtls-domain -f - <<'EOF'
{
  "cert": "<NEW_CERT>",
  "key": "<NEW_KEY>",
  "client": {
    "ca": "<NEW_OR_SAME_CA>"
  }
}
EOF

APISIX picks up the new certificate immediately — no restart needed.

Config Sync Example

version: "1"
ssls:
  - id: api-mtls
    cert: |
      -----BEGIN CERTIFICATE-----
      <server certificate>
      -----END CERTIFICATE-----
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      <server private key>
      -----END RSA PRIVATE KEY-----
    snis:
      - api.example.com
    client:
      ca: |
        -----BEGIN CERTIFICATE-----
        <CA certificate for client verification>
        -----END CERTIFICATE-----
upstreams:
  - id: secure-backend
    type: roundrobin
    scheme: https
    nodes:
      "backend:443": 1
    tls:
      client_cert: |
        -----BEGIN CERTIFICATE-----
        <client certificate for upstream>
        -----END CERTIFICATE-----
      client_key: |
        -----BEGIN RSA PRIVATE KEY-----
        <client private key for upstream>
        -----END RSA PRIVATE KEY-----
routes:
  - id: mtls-api
    uri: /api/*
    host: api.example.com
    upstream_id: secure-backend

Troubleshooting

Symptom	Cause	Fix
SSL handshake failure (client side)	Client cert not signed by the CA in client.ca	Verify CA chain; check that client cert is signed by the correct CA
"no required SSL certificate"	Client didn't send a certificate	Configure client to present cert (--cert in curl)
502 to upstream	Upstream rejects APISIX's client cert	Verify tls.client_cert is signed by the upstream's trusted CA
Certificate expired	TLS cert past validity date	Rotate certificate with a6 ssl update
SNI mismatch	Domain doesn't match snis list	Add the domain to the snis array
"unable to verify"	Self-signed cert without proper CA trust	Use --cacert in curl or add CA to system trust store
Mixed HTTP/HTTPS	Route accessible on both ports	Configure APISIX listen to only expose HTTPS port for mTLS domains

---

This page is generated from a6-recipe-mtls/SKILL.md in the api7/a6 repository. Browse all skills on the AI Agent Skills page.