Skip to main content

Version: 3.2.16.3

Performance Testing Benchmarks

This document will explain the testing environments, methodologies, and results, to help you evaluate how API7 Enterprise performs under various load conditions and how to conduct performance testings yourself.

In addition to referring to the performance testing benchmarks, you can also conduct your own performance tests with the setup in the performance benchmark repository. This repository documents in detail all the resource deployment configurations used for testing as well as the specific configuration information for each test scenario. Through this repository, you can conduct performance benchmark tests on your own deployment of API7 Gateway following the provided guidelines. Before starting the tests, please ensure that the performance baselines you are getting are roughly consistent with the test results.

Methodologies

  • Environment: Kubernetes.
  • Test Scenarios:
    1. Only enable the mocking plugin to obtain the baseline performance value of API7 Gateway. This plugin returns mock data in a specified format, and requests are not forwarded to upstream servers;
    2. No plugins enabled;
    3. Only enable the limit-count rate limiting plugin;
    4. Only enable the key-auth authentication plugin;
    5. Enable both the key-auth and limit-count plugins;
  • Routes and Consumers:
    1. Single route and a single consumer;
    2. 100 routes and 100 consumers;
  • Sample size: Test Data Collection: Each test case is run 5 times, with each run lasting 2 minutes. The reported results are the average of the 5 test runs.

Performance Benchmarking Results

Test ScenariosNumber of Routes/ConsumersForward to UpstreamQPSP99 (MS)P95 (MS)
Only enable the mocking1 route, 0 consumersFalse310,392.071.161.08
No plugins enabled1 route, 0 consumersTrue167,019.372.302.16
No plugins enabled100 routes, 0 consumersTrue162,753.172.312.16
Only enable the limit-count1 route, 0 consumersTrue145,370.102.432.24
Only enable the limit-count100 routes, 0 consumersTrue143,108.402.452.25
Only enable the key-auth1 route, 1 consumerTrue147,869.492.412.22
Only enable the key-auth100 routes, 100 consumersTrue145,070.932.432.25
Enable both the key-auth and limit-count1 route, 1 consumerTrue136,725.472.432.26
Enable both the key-auth and limit-count100 routes, 100 consumersTrue133,782.952.482.30

Test Environment

This performance test was conducted in the AWS EKS environment. When building the test environment, it is essential to ensure that API7 Gateway, NGINX Upstream, and the performance testing tool wrk2 are deployed on separate nodes, all using c5.4xlarge EC2 instances. This configuration ensures a reasonable allocation of resources and avoids resource contention issues.

During stress testing, it is recommended to use the top command or other system monitoring tools to monitor the process resource utilization of API7 Gateway and NGINX Upstream servers in real-time. This ensures that each test reaches the performance bottleneck of API7 Gateway, resulting in accurate and reliable performance test results.

Here is an overview of the main components involved in this test and their specifications:

NameDetails
NodeAmazon Linux2 (AL2_x86_64)
Kubernetes1.29
API7 Gateway3.2.13.0
Upstream Servicenginx/1.25.4
Performance Benchmark Toolwrk2

Deployment Topology

K8s deploy

Test Configuration

For this test, the number of worker_processes was adjusted to match the available virtual core count (i.e. 16 vCPUs) on the node running API7 Gateway. No further modifications were made to the system configuration.

Additional Information


API7.ai Logo

API Management for Modern Architectures with Edge, API Gateway, Kubernetes, and Service Mesh.

Product

API7 Cloud

SOC2 Type IIISO 27001HIPAAGDPRRed Herring

Copyright © APISEVEN Ltd. 2019 – 2024. Apache, Apache APISIX, APISIX, and associated open source project names are trademarks of the

Apache Software Foundation