DevOps with API7 Enterprise Gateway, Declarative Configurations, and Pipelines

Introduction

This document guides you on how to set up API7 Enterprise with Declarative Configurations Tool (ADC, APISIX/API7 Declarative Configuration CLI). To automate the process, use the adc.yaml file as the only source of truth to leverage GitOps.

ADC Introduction

ADC takes the adc.yaml file as the only source and transforms configurations into API Gateways accordingly. In other words, API gateways have the same state as described in adc.yaml.

Steps

1. Configure ADC and Connect ADC to Gateway Instances
2. Publish Configurations, Proxy Requests, and Enable Key Authentication with Consumers
3. Modify Configurations and Apply Changes

Notes

1. In this document, you will use API7 Enterprise Gateway 3.2.11.1, which is based on Apache APISIX 3.2 and does not yet provide Configuration Validation APIs. Therefore, you cannot utilize the ADC validate command. This feature is available starting from Apache APISIX v3.5+.
2. ADC supports loading your OpenAPI 3.0 spec file and converting it to adc.yaml. However, you should update the converted file accordingly. For example, you need to add Authentication policies and bind a Service ID. API7 have plans to improve these functionalities for better user experience.
3. Enterprises may have different Git/SVN version control services and firewall policies. However, as long as the ADC tool can access Gateway's Admin APIs, it should work. This document provides a demonstration of how to operate the ADC on a local computer.

Prerequisites

• API7 Enterprise: 3.2.11.1
• ADC: 0.7.3

Deploy API7 Enterprise

API7 Enterprise supports Docker and Kubernetes. You can have different deployment methods based on your requirements.

After deploying API7 Enterprise, you should have the following components and information:

• Dashboard Component: 7443/7080 Port
• API7 Gateway Component: http://152.42.234.39:9080
• Admin APIs: https://152.42.234.39:7443

Download ADC

• https://run.api7.ai/adc/release/adc_0.7.3_linux_amd64.tar.gz
• https://run.api7.ai/adc/release/adc_0.7.3_linux_arm64.tar.gz
• https://run.api7.ai/adc/release/adc_0.7.3_darwin_arm64.tar.gz

$ ./adc -h

Usage: adc [options] [command]

Options:
  -V, --version   output the version number
  -h, --help      display help for command

Commands:
  ping [options]  Verify connectivity with backend
  dump [options]  Dump configurations from the backend
  diff [options]  Show the difference between local and backend configurations
  sync [options]  Sync local configurations to backend
  convert         Convert other API spec to ADC configurations
  lint [options]  Check provided configuration files, local execution only, ensuring inputs meet ADC requirements
  help [command]  display help for com

Steps

Generate API Token

ADC needs an API Token to access Admin APIs. You can generate an API Token from the Dashboard or through Admin APIs.

• Generate API Token from Dashboard: Login -> Organization -> Tokens -> Generate New Token
• Generate API Token from Admin APIs

Configure ADC Credentials

1. Create a .env file in the same directory as the ADC binary.

This document creates a .env file to store the API Token and Admin API Endpoint. You can also pass these values as flags or stored in GitHub Action, Jenkins, or GitLab.

ADC_BACKEND=api7ee
ADC_SERVER=https://152.42.234.39:7443
ADC_TOKEN=a7ee-6baF8488i8wJ5aj7mEo3BT705573eC8GH905qGrdn889zUWcR37df66a34e9954b61918c5dfd13abce3e

2. Verify Admin APIs are accessible.

$ ./adc ping

Connected to backend successfully!

Publish Configurations

1. Create the adc.yaml file:

services:
  - name: SOAP Service
    description: "This Service is for SOAP requests"
    upstream:
      name: default
      scheme: https
      type: roundrobin
      hash_on: vars
      nodes:
        - host: apps.learnwebservices.com
          port: 443
          weight: 1
          priority: 0
      retry_timeout: 0
      pass_host: pass
    routes:
      - uris:
          - /services/hello
        name: SOAP proxy
        methods:
          - POST
      - uris:
          - /SayHello
        name: REST to SOAP
        methods:
          - POST
        plugins:
          soap:
            wsdl_url: https://apps.learnwebservices.com/services/hello?wsdl
  - name: httpbin Service
    description: "This is the httpbin Services proxy service"
    labels:
      app: httpbin
      custom_key: custom_value
    routes:
      - name: Generate UUID
        methods:
          - GET
        uris:
          - /uuid
      - name: Anything
        methods:
          - GET
        uris:
          - /anything
        plugins:
          key-auth:
            _meta:
              disable: false
      - name: Rate Limited IP
        methods:
          - GET
        uris:
          - /ip
        plugins:
          limit-count:
            _meta:
              disable: false
            count: 2
            time_window: 10
            rejected_code: 429
            policy: local
    upstream:
      name: default
      scheme: http
      type: roundrobin
      hash_on: vars
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1
          priority: 0
      retry_timeout: 0
      pass_host: pass
  - name: SSE Service
    labels:
      service: sse
    upstream:
      scheme: https
      nodes:
        - host: www.esegece.com
          port: 2053
          weight: 1
          priority: 0
      type: roundrobin
      pass_host: node
    plugins:
      proxy-buffering:
        disable_proxy_buffering: true
    routes:
      - name: SSE API
        uris:
          - /sse
        methods:
          - GET
consumers:
  - username: tom
    plugins:
      key-auth:
        key: tomskey
ssls: []
global_rules: {}

2. Sync adc.yaml to gateway instances

$ ./adc sync -f adc.yaml

✔ Load local configuration
✔ Load remote configuration
✔ Diff configuration
✔ Sync configuration

3. Proxy Validation

$ curl 152.42.234.39:9080/uuid -v

*   Trying 152.42.234.39:9080...
* Connected to 152.42.234.39 (152.42.234.39) port 9080
> GET /uuid HTTP/1.1
> Host: 152.42.234.39:9080
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 53
< Connection: keep-alive
< Date: Wed, 17 Apr 2024 09:56:22 GMT
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Credentials: true
< Server: API7/3.2.8
<

{
  "uuid": "22b888f4-9e96-4d09-93a2-408b14e772fe"
}

* Connection #0 to host 152.42.234.39 left intact

4. Plugins Validation: Key Authentication

$ curl 152.42.234.39:9080/anything

{"message":"Missing API key found in request"}

$ curl 152.42.234.39:9080/anything -H "apikey: tomskey"

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Apikey": "tomskey", 
    "Host": "152.42.234.39", 
    "User-Agent": "curl/8.4.0", 
    "X-Amzn-Trace-Id": "Root=1-661f9d57-42c4a66b07b361713713da44", 
    "X-Forwarded-Host": "152.42.234.39"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "182.255.32.50, 152.42.234.39", 
  "url": "http://152.42.234.39/anything"
}

Modify Configurations and Apply Changes

1. Update the adc.yaml to the following configurations (added a new route /headers for the httpbin service):

services:
  - name: SOAP Service
    description: ""
    upstream:
      name: default
      scheme: https
      type: roundrobin
      hash_on: vars
      nodes:
        - host: apps.learnwebservices.com
          port: 443
          weight: 1
          priority: 0
      retry_timeout: 0
      pass_host: pass
    routes:
      - uris:
          - /services/hello
        name: SOAP proxy
        methods:
          - POST
      - uris:
          - /SayHello
        name: REST to SOAP
        methods:
          - POST
        plugins:
          soap:
            wsdl_url: https://apps.learnwebservices.com/services/hello?wsdl
  - name: httpbin Service
    description: ""
    labels:
      app: httpbin
      custom_key: custom_value
    routes:
      - name: Get Headers
        methods:
          - GET
        uris:
          - /headers
      - name: Generate UUID
        methods:
          - GET
        uris:
          - /uuid
      - name: Anything
        methods:
          - GET
        uris:
          - /anything
        plugins:
          key-auth:
            _meta:
              disable: false
      - name: Rate Limited IP
        methods:
          - GET
        uris:
          - /ip
        plugins:
          limit-count:
            _meta:
              disable: false
            count: 2
            time_window: 10
            rejected_code: 429
            policy: local
    upstream:
      name: default
      scheme: http
      type: roundrobin
      hash_on: vars
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1
          priority: 0
      retry_timeout: 0
      pass_host: pass
  - name: SSE Service
    labels:
      service: sse
    upstream:
      scheme: https
      nodes:
        - host: www.esegece.com
          port: 2053
          weight: 1
          priority: 0
      type: roundrobin
      pass_host: node
    plugins:
      proxy-buffering:
        disable_proxy_buffering: true
    routes:
      - name: SSE API
        uris:
          - /sse
        methods:
          - GET
consumers:
  - username: tom
    plugins:
      key-auth:
        key: tomskey
ssls: []
global_rules: {}

2. Determine the differences between local configurations and remote configurations.

info

ADC is in actively development, and the diff algorithm will be improved and optimized.

$ ./adc diff -f ./adc.yaml

✔ Load local configuration
✔ Load remote configuration
✔ Diff configuration
  › update consumer: "tom"
    update route: "REST to SOAP"
    update route: "SOAP proxy"
    update route: "Rate Limited IP"
    update route: "Anything"
    update route: "Generate UUID"
    create route: "Get Headers"
    update service: "SSE Service"
    update route: "SSE API"
    Summary: 1 will be created, 8 will be updated, 0 will be deleted
✔ Write detail diff result to file

This command also generates a diff.yaml file with detailed differences.

- resourceType: consumer
  type: update
  resourceId: tom
  resourceName: tom
  oldValue:
    username: tom
    description: ""
    plugins:
      key-auth:
        key: tomskey
  newValue:
    username: tom
    plugins:
      key-auth:
        key: tomskey
  diff:
    added: {}
    deleted: {}
    updated: {}
- resourceType: route
  type: update
  resourceId: bef0a3351a392e74c960f9a58c1d025d803f2aef
  resourceName: REST to SOAP
  oldValue:
    uris:
      - /SayHello
    name: REST to SOAP
    methods:
      - POST
    enable_websocket: false
    plugins:
      soap:
        wsdl_url: https://apps.learnwebservices.com/services/hello?wsdl
  newValue:
    uris:
      - /SayHello
    name: REST to SOAP
    methods:
      - POST
    plugins:
      soap:
        wsdl_url: https://apps.learnwebservices.com/services/hello?wsdl
  diff:
    added: {}
    deleted: {}
    updated: {}
  parentId: 602dfcf4c39218f87d40c5d1df8b531b49ca88e8
- resourceType: route
  type: update
  resourceId: da37e65d428446d156279156ac3248c00d0a1533
  resourceName: SOAP proxy
  oldValue:
    uris:
      - /services/hello
    name: SOAP proxy
    methods:
      - POST
    enable_websocket: false
  newValue:
    uris:
      - /services/hello
    name: SOAP proxy
    methods:
      - POST
  diff:
    added: {}
    deleted: {}
    updated: {}
  parentId: 602dfcf4c39218f87d40c5d1df8b531b49ca88e8
- resourceType: route
  type: update
  resourceId: b586591b59c13461ed8932228cb23e53040c09d4
  resourceName: Rate Limited IP
  oldValue:
    uris:
      - /ip
    name: Rate Limited IP
    methods:
      - GET
    enable_websocket: false
    plugins:
      limit-count:
        _meta:
          disable: false
        count: 2
        policy: local
        rejected_code: 429
        time_window: 10
  newValue:
    name: Rate Limited IP
    methods:
      - GET
    uris:
      - /ip
    plugins:
      limit-count:
        _meta:
          disable: false
        count: 2
        time_window: 10
        rejected_code: 429
        policy: local
  diff:
    added: {}
    deleted: {}
    updated: {}
  parentId: 5ce4033cfe1015450e0b81186f7d54b9327cc302
- resourceType: route
  type: update
  resourceId: 5f2de5df1a292b4c8a73f0ec23271233d75707c6
  resourceName: Anything
  oldValue:
    uris:
      - /anything
    name: Anything
    methods:
      - GET
    enable_websocket: false
    plugins:
      key-auth:
        _meta:
          disable: false
  newValue:
    name: Anything
    methods:
      - GET
    uris:
      - /anything
    plugins:
      key-auth:
        _meta:
          disable: false
  diff:
    added: {}
    deleted: {}
    updated: {}
  parentId: 5ce4033cfe1015450e0b81186f7d54b9327cc302
- resourceType: route
  type: update
  resourceId: ed048a2f75fe33eab67319810fbb94bb778d7d97
  resourceName: Generate UUID
  oldValue:
    uris:
      - /uuid
    name: Generate UUID
    methods:
      - GET
    enable_websocket: false
  newValue:
    name: Generate UUID
    methods:
      - GET
    uris:
      - /uuid
  diff:
    added: {}
    deleted: {}
    updated: {}
  parentId: 5ce4033cfe1015450e0b81186f7d54b9327cc302
- resourceType: route
  resourceId: 6b124aff482499cbf7bdad5a56b13205b24ba58e
  resourceName: Get Headers
  type: create
  newValue:
    name: Get Headers
    methods:
      - GET
    uris:
      - /headers
  parentId: 5ce4033cfe1015450e0b81186f7d54b9327cc302
- resourceType: service
  type: update
  resourceId: 54154d3cdf6379ab8686890d27fabb6bf8fa3ace
  resourceName: SSE Service
  oldValue:
    name: SSE Service
    description: ""
    labels:
      service: sse
    upstream:
      name: default
      scheme: https
      type: roundrobin
      hash_on: vars
      nodes:
        - host: www.esegece.com
          port: 2053
          weight: 1
          priority: 0
      retry_timeout: 0
      pass_host: node
    plugins:
      proxy-buffering:
        disable_proxy_buffering: true
  newValue:
    name: SSE Service
    labels:
      service: sse
    upstream:
      scheme: https
      nodes:
        - host: www.esegece.com
          port: 2053
          weight: 1
          priority: 0
      type: roundrobin
      pass_host: node
    plugins:
      proxy-buffering:
        disable_proxy_buffering: true
  diff:
    added: {}
    deleted:
      upstream: {}
    updated: {}
  subEvents:
    - &a1
      resourceType: route
      type: update
      resourceId: 6b808fa543c7c3391321b813d0dc2d658ab02c10
      resourceName: SSE API
      oldValue:
        uris:
          - /sse
        name: SSE API
        methods:
          - GET
        enable_websocket: false
      newValue:
        name: SSE API
        uris:
          - /sse
        methods:
          - GET
      diff:
        added: {}
        deleted: {}
        updated: {}
      parentId: 54154d3cdf6379ab8686890d27fabb6bf8fa3ace
- *a1

3. Apply Changes

$ ./adc sync -f ./adc.yaml

✔ Load local configuration
✔ Load remote configuration
✔ Diff configuration
✔ Sync configuration

4. Validation

Note: Because you have added a new route /headers for the httpbin service, you can access the new route and it should return the headers.

$ curl 152.42.234.39:9080/headers

{
  "headers": {
    "Accept": "*/*", 
    "Host": "152.42.234.39", 
    "User-Agent": "curl/8.4.0", 
    "X-Amzn-Trace-Id": "Root=1-661f9edd-188a0adf5d3bf0a509f5b034", 
    "X-Forwarded-Host": "152.42.234.39"
  }
}

Conclusion

This document demonstrates how to use the ADC tool to publish configurations, proxy requests, and enable key authentication with consumers. You can also modified the configurations and applied changes to the Gateway instances. The ADC tool is a powerful tool that can help you manage your API Gateway configurations efficiently.

When integrating ADC with your CI/CD pipeline, you can automate the process of publishing configurations and validating them. This can help you ensure that your configurations are correct and that your APIs are secure.