Getting Started

Overview

The published benchmark records one high-load test at 5,000 requests per second (RPS) on two AWS EC2 instance types. It is evidence for those test conditions, not a production capacity guarantee.

Observed highlights:

• Success Rate: 100% in the recorded 5,000-RPS runs
• Gateway overhead: 59µs on t3.medium and 11µs on t3.xlarge under the documented measurement method
• Resource use: recorded throughput and memory stayed within each instance during the test window

---

Test Environment Summary

DeepIntShield was benchmarked on two primary AWS EC2 instance configurations:

t3.medium (2 vCPUs, 4GB RAM)

• Buffer Size: 15,000
• Initial Pool Size: 10,000
• Use Case: Cost-effective option for moderate workloads

t3.xlarge (4 vCPUs, 16GB RAM)

• Buffer Size: 20,000
• Initial Pool Size: 15,000
• Use Case: High-performance option for demanding workloads

---

Performance Comparison at a Glance

Metric	t3.medium	t3.xlarge	Improvement
Success Rate @ 5k RPS	100%	100%	No failed requests
DeepIntShield Overhead	59 µs	11 µs	-81%
Average Latency	2.12s	1.61s	-24%
Peak Memory Usage	1,312.79 MB	3,340.44 MB	+155%

Note: t3.xlarge tests used significantly larger response payloads (~10 KB vs ~1 KB), yet still achieved better performance metrics.

Note

All results use mocked OpenAI calls. Provider behavior, guardrail policy, networking, CPU-credit state, test duration, and payload mix materially affect results. The historical pages do not identify a release SHA and test date, so reproduce them with the current build before capacity planning or setting an SLO.

---

Configuration Flexibility

One of DeepIntShield’s key strengths is its configuration flexibility. You can fine-tune the speed ↔ memory trade-off based on your specific requirements:

Configuration Parameter	Effect
initial_pool_size	Higher values = faster performance, more memory usage
buffer_size & concurrency	Controls queue depth and max parallel workers (per provider)
retry & timeout	Tune aggressiveness for each provider to meet your SLOs

Configuration Philosophy:

• Higher settings (like t3.xlarge profile) prioritize raw speed
• Lower settings (like t3.medium profile) optimize for memory efficiency
• Custom tuning lets you find the sweet spot for your specific workload

---

Next Steps

Detailed Performance Analysis

• t3.medium Performance - Deep dive into cost-effective performance
• t3.xlarge Performance - High-performance configuration analysis

Run Your Own Tests

• Run Your Own Benchmarks - Step-by-step guide to benchmark DeepIntShield in your environment

Ready to dive deeper? Choose your instance type above or learn how to run your own performance tests.