Performance Testing Benchmarks
This document will explain the testing environments, methodologies, and results, to help you evaluate how API7 Enterprise performs under various load conditions and how to conduct performance testings yourself.
In addition to referring to the performance testing benchmarks, you can also conduct your own performance tests with the setup in the performance benchmark repository. This repository documents in detail all the resource deployment configurations used for testing as well as the specific configuration information for each test scenario. Through this repository, you can conduct performance benchmark tests on your own deployment of API7 Gateway following the provided guidelines. Before starting the tests, please ensure that the performance baselines you are getting are roughly consistent with the test results.
Methodologies
- Environment: Kubernetes.
- Test Scenarios:
- Only enable the
mocking
plugin to obtain the baseline performance value of API7 Gateway. This plugin returns mock data in a specified format, and requests are not forwarded to upstream servers; - No plugins enabled;
- Only enable the
limit-count
rate limiting plugin; - Only enable the
key-auth
authentication plugin; - Enable both the
key-auth
andlimit-count
plugins;
- Only enable the
- Routes and Consumers:
- Single route and a single consumer;
- 100 routes and 100 consumers;
- Sample size: Test Data Collection: Each test case is run 5 times, with each run lasting 2 minutes. The reported results are the average of the 5 test runs.
Performance Benchmarking Results
- AWS EKS
Test Scenarios | Number of Routes/Consumers | Forward to Upstream | QPS | P99 (MS) | P95 (MS) |
---|---|---|---|---|---|
Only enable the mocking | 1 route, 0 consumers | False | 310,392.07 | 1.16 | 1.08 |
No plugins enabled | 1 route, 0 consumers | True | 167,019.37 | 2.30 | 2.16 |
No plugins enabled | 100 routes, 0 consumers | True | 162,753.17 | 2.31 | 2.16 |
Only enable the limit-count | 1 route, 0 consumers | True | 145,370.10 | 2.43 | 2.24 |
Only enable the limit-count | 100 routes, 0 consumers | True | 143,108.40 | 2.45 | 2.25 |
Only enable the key-auth | 1 route, 1 consumer | True | 147,869.49 | 2.41 | 2.22 |
Only enable the key-auth | 100 routes, 100 consumers | True | 145,070.93 | 2.43 | 2.25 |
Enable both the key-auth and limit-count | 1 route, 1 consumer | True | 136,725.47 | 2.43 | 2.26 |
Enable both the key-auth and limit-count | 100 routes, 100 consumers | True | 133,782.95 | 2.48 | 2.30 |
Test Environment
This performance test was conducted in the AWS EKS environment. When building the test environment, it is essential to ensure that API7 Gateway, NGINX Upstream, and the performance testing tool wrk2
are deployed on separate nodes, all using c5.4xlarge
EC2 instances. This configuration ensures a reasonable allocation of resources and avoids resource contention issues.
During stress testing, it is recommended to use the top
command or other system monitoring tools to monitor the process resource utilization of API7 Gateway and NGINX Upstream servers in real-time. This ensures that each test reaches the performance bottleneck of API7 Gateway, resulting in accurate and reliable performance test results.
Here is an overview of the main components involved in this test and their specifications:
Name | Details |
---|---|
Node | Amazon Linux2 (AL2_x86_64) |
Kubernetes | 1.29 |
API7 Gateway | 3.2.13.0 |
Upstream Service | nginx/1.25.4 |
Performance Benchmark Tool | wrk2 |
Deployment Topology
Test Configuration
For this test, the number of worker_processes
was adjusted to match the available virtual core count (i.e. 16 vCPUs) on the node running API7 Gateway. No further modifications were made to the system configuration.
Additional Information
- Establish Performance Benchmark: Find out how to optimize the performance of API7 Gateway.
- Guide for Performance Testing of API7 Gateway on AWS EKS: View detailed steps and guidelines on how to establish performance benchmark on AWS EKS.