Interactive AI Serving Guide: Instant PTU/Priority, Goldilocks, Flex & -50% Batch

PTU + Priority

1. Instant Real-Time

< 0.5s • 0% 429 Errors

Voice, Live IDE, Support

-90% KV Cache

2. Goldilocks Zone

1-2s • 40-80 TPS

Chat, RAG, Drafting

-50% OFF (Flex)

3. Flex & Reasoning

5-60s • N* Capped

CI/CD, PR Review, Agents

-50% OFF (Batch)

4. Bulk Batch API

< 24h • Free Quota

Nightly Audits, Evals, ETL

⚡ Instant Real-Time Playbook: Zero HTTP 429 rate-limit drops

PTU Base + Priority Spillover

            
1. Steady Base
Reserved PTU / GSU
Dedicated GPU (0% 429s)

➔

            
2. Spike Shield
Priority Spillover
Bursts route to multi-cloud

➔

            
3. Unit Outcome
< 0.4s • ~0.75x Cost
Zero idle peak PTU waste

🌙 Batch / Flex Offload (-50%) 50%

⚡ KV Cache Hit Rate (-95%) 90%

Monthly ($10k Base)

$2,450/mo

Saved

            -75.5%
          