Performance Testing Benchmarks

This document will explain the testing environments, methodologies, and results, to help you evaluate how API7 Enterprise performs under various load conditions and how to conduct performance testings yourself.

In addition to referring to the performance testing benchmarks, you can also conduct your own performance tests with the setup in the performance benchmark repository. This repository documents in detail all the resource deployment configurations used for testing as well as the specific configuration information for each test scenario. Through this repository, you can conduct performance benchmark tests on your own deployment of API7 Gateway following the provided guidelines. Before starting the tests, please ensure that the performance baselines you are getting are roughly consistent with the test results.

Methodologies

• Environment: Kubernetes.
• Test Scenarios:
  1. Only enable the mocking plugin to obtain the baseline performance value of API7 Gateway. This plugin returns mock data in a specified format, and requests are not forwarded to upstream servers;
  2. No plugins enabled;
  3. Only enable the limit-count rate limiting plugin;
  4. Only enable the key-auth authentication plugin;
  5. Enable both the key-auth and limit-count plugins;
• Routes and Consumers:
  1. Single route and a single consumer;
  2. 100 routes and 100 consumers;
• Sample size: Test Data Collection: Each test case is run 5 times, with each run lasting 2 minutes. The reported results are the average of the 5 test runs.

Performance Benchmarking Results

AWS EKS

Test Scenarios	Number of Routes/Consumers	Forward to Upstream	QPS	P99 (MS)	P95 (MS)
Only enable the mocking	1 route, 0 consumers	False	310,392.07	1.16	1.08
No plugins enabled	1 route, 0 consumers	True	167,019.37	2.30	2.16
No plugins enabled	100 routes, 0 consumers	True	162,753.17	2.31	2.16
Only enable the limit-count	1 route, 0 consumers	True	145,370.10	2.43	2.24
Only enable the limit-count	100 routes, 0 consumers	True	143,108.40	2.45	2.25
Only enable the key-auth	1 route, 1 consumer	True	147,869.49	2.41	2.22
Only enable the key-auth	100 routes, 100 consumers	True	145,070.93	2.43	2.25
Enable both the key-auth and limit-count	1 route, 1 consumer	True	136,725.47	2.43	2.26
Enable both the key-auth and limit-count	100 routes, 100 consumers	True	133,782.95	2.48	2.30

Test Environment

This performance test was conducted in the AWS EKS environment. When building the test environment, it is essential to ensure that API7 Gateway, NGINX Upstream, and the performance testing tool wrk2 are deployed on separate nodes, all using c5.4xlarge EC2 instances. This configuration ensures a reasonable allocation of resources and avoids resource contention issues.

During stress testing, it is recommended to use the top command or other system monitoring tools to monitor the process resource utilization of API7 Gateway and NGINX Upstream servers in real-time. This ensures that each test reaches the performance bottleneck of API7 Gateway, resulting in accurate and reliable performance test results.

Here is an overview of the main components involved in this test and their specifications:

Name	Details
Node	Amazon Linux2 (AL2_x86_64)
Kubernetes	1.29
API7 Gateway	3.2.13.0
Upstream Service	nginx/1.25.4
Performance Benchmark Tool	wrk2

Deployment Topology

Test Configuration

For this test, the number of worker_processes was adjusted to match the available virtual core count (i.e. 16 vCPUs) on the node running API7 Gateway. No further modifications were made to the system configuration.

Additional Information

• Establish Performance Benchmark: Find out how to optimize the performance of API7 Gateway.
• Guide for Performance Testing of API7 Gateway on AWS EKS: View detailed steps and guidelines on how to establish performance benchmark on AWS EKS.