Routes, keys and limits
How the gateway chooses a target, what happens on errors, and how keys enforce budgets and rate limits.
Who does what
The path of a model request
- 01
Authenticate
The application calls a route with a scoped team or application key.
Your application
- 02
Check policy
The gateway checks routes, budgets and rate limits and applies configured redaction.
AI gateway
- 03
Select target
Use the route strategy, permitted regions and configured fallback targets.
AI gateway
- 04
Return + meter
Return the provider response and record request metadata, tokens and cost.
Gateway + tokenomics
The gateway is in the request path. Direct calls to providers bypass gateway controls.
Strategies
| Strategy | Behaviour |
|---|---|
priority | Use targets in order. A target with until.utilization receives traffic until its provisioned utilization reaches the limit, then the next target is used. |
lowest-cost | Send each request to the cheapest healthy target, using your contracted prices. |
lowest-latency | Send each request to the target with the lowest recent latency. |
weighted | Split traffic by weight, for gradual migrations and comparisons. |
Fallback and retries
Targets with on: [429, 5xx, timeout] receive a request only when the previous target throttles, fails or times out. Retries use exponential backoff and stop at retries.attempts. Every fallback is recorded with its reason.
Keys
- Each key has a name, team, product and the routes it may call.
budget.action: warnnotifies the owner at thresholds;hard-stopreturns HTTP 429 with a clear message once the monthly budget is spent.limits.tokensPerMinuteandlimits.requestsPerMinuteare enforced across all gateway replicas.- Keys can be rotated without downtime; the old key keeps working for the overlap you choose.
Cache, redaction and residency
cache.mode: exactreturns stored responses for identical requests within the TTL. Cached tokens are counted and cost nothing.redactmasks the listed entity types in prompts before they leave the gateway.residencylimits targets to providers and regions in the US or EU.
Where the gateway runs
The gateway is managed by UnitVane in the region you choose. It records request metadata, never prompt content unless logging is enabled for a route. There is no self-hosted gateway.