t3.xlarge

​

Instance Configuration

• vCPUs: 4
• Memory: 16GB RAM
• Network Performance: Up to 5 Gigabit

• Buffer Size: 20,000
• Initial Pool Size: 15,000
• Test Load: 5,000 requests per second (RPS)

​

Performance Results

​

Overall Performance Metrics

Note: t3.xlarge tests used significantly larger response payloads (~10 KB vs ~1 KB on t3.medium) to stress-test performance with realistic production data sizes.

​

Detailed Performance Breakdown

​

Performance Analysis

​

Exceptional Performance Improvements

1. Dramatic Overhead Reduction: 81% lower Bifrost overhead (59 µs → 11 µs)
2. Superior Queue Management: 96% faster queue wait times (47.13 µs → 1.67 µs)
3. Faster JSON Processing: 58% improvement in marshaling despite larger payloads
4. Efficient Response Parsing: 81% faster parsing even with 7.5x larger responses
5. Perfect Reliability: 100% success rate maintained under high load

​

Resource Utilization

• Memory Efficiency: Uses only 21% of available RAM (3,340.44 MB / 16GB)
• CPU Performance: Excellent multi-core utilization for 5,000 RPS
• Headroom: Substantial capacity for traffic spikes and growth

​

Performance Characteristics

• Key Selection: 10 ns (near-instantaneous)
• Error Handling: 162 ns
• Params Preparation: 417 ns

• HTTP Request: 1.50s (external provider call)
• Response Parsing: 2.11 ms (handling 10 KB responses)
• JSON Marshaling: 26.80 µs

​

Scalability and Headroom

​

Exceptional Scaling Characteristics

• Memory: 21% used (13GB available headroom)
• Queue Performance: 1.67 µs wait time (near-optimal)
• Processing Speed: Sub-microsecond for most operations

• Traffic Spikes: Can likely handle 15,000+ RPS bursts
• Response Size Growth: Efficiently handles 10 KB responses
• Concurrent Users: Supports thousands of simultaneous users

​

Ideal Use Cases

• High-throughput production workloads (5,000+ RPS sustained)
• Enterprise applications with strict SLA requirements
• Large response payloads (documents, code generation, etc.)
• Mission-critical systems requiring sub-15µs overhead
• Applications with traffic spikes needing substantial headroom

​

Advanced Configuration

​

Optimal Settings for t3.xlarge

{
  "client": {
    "initial_pool_size": 15000,
    "buffer_size": 20000
  }
}

​

Performance Tuning Opportunities

• Increase initial_pool_size to 18,000-20,000
• Increase buffer_size to 25,000-30,000
• Trade-off: Higher memory usage (still well within limits)

• Current config already very efficient at 21% RAM usage
• Could reduce settings if needed, but performance gains would be lost

• Consider initial_pool_size up to 25,000
• Increase buffer_size to 35,000+
• Monitor memory usage approaching 50% of available RAM

​

Performance Comparison

​

vs. t3.medium Performance

• 81% overhead reduction while handling 7.5x larger responses
• Exceptional efficiency with only 21% memory utilization
• Dramatic queue performance improvements
• Substantial headroom for growth and traffic spikes

​

Next Steps

• Run Your Own Benchmarks with your specific payload sizes
• Compare with t3.medium for cost-optimization analysis