Portkey AI Gateway review
TypeScript LLM gateway with one OpenAI-style API, retries, fallbacks, load balancing and guardrail hooks
Data refreshedMethodologyHow we testedChandan Kumar is a founder of Geekflare. He is a technology enthusiast and entrepreneur who loves to help businesses and people around the world. Chandan has worked at BNP Paribas, Citibank, Deutsche Bank, Motorola, and HP. He’s got an in-depth understanding of business software and IT infrastructure.
Verdict
Portkey's gateway is a small TypeScript service with a wide provider list and simple JSON configs for retries, fallbacks, load balancing and guardrails, and it runs on Node.js, Bun, Docker or Cloudflare Workers. The catch is momentum: the last release was in January 2026, nothing has been merged since May, and the 2.0 release that was meant to open source the enterprise gateway has not shipped since Portkey became part of Palo Alto Networks.
Best for
Teams that want a light routing and guardrail proxy they can run anywhere, including at the edge
Not for
Teams that need budgets, spend tracking or admin logins in the free version
Pros
- Retries, fallbacks, load balancing and conditional routing from one JSON config
- Runs on Node.js, Bun, Docker or Cloudflare Workers
- Built-in guardrail checks plus partner plugins
- MIT licence with no licence-key features
Cons
- No release since January 2026 and no merges since May 2026
- Semantic caching, spend analytics, RBAC and SSO are in paid plans
- The released 1.15.2 shows prompts and provider keys on an open log stream unless NODE_ENV is production
Ask SparkianIs Portkey AI Gateway right for your project?
Describe what you need and AI will check the fit against this review.
Portkey AI Gateway features and capabilities
Fallbacks, retries and load balancing
A config can retry a failed call up to five times with backoff, fall back to another provider on the errors you choose, and spread traffic across keys or providers by weight.
Guardrails on input and output
Checks such as word lists, regex and JSON schema run before or after the model call and can block a reply, and partner plugins add services like Pangea, Aporia and Prisma AIRS.
Rate limits per provider
Each provider integration in conf.json can carry request and token limits per hour, shared across nodes when Redis is configured.
Runs at the edge or in a container
The same code runs as a Cloudflare Worker, a Node.js or Bun process or a Docker container, so the gateway can sit close to users or inside your own network.
Local request console
A built-in page shows live request logs while you develop, though stored logs and spend analytics are only in the hosted app.
- FallbacksSwitches to another model or provider when a call fails
- Yes
- Load balancingSpreads traffic across keys, deployments or providers
- Yes
- Budgets and limitsSpending caps or rate limits per key, user or team
- PartlyRequest and token limits per provider in conf.json; spend budgets per key are paid
- Cost trackingRecords spend per request and per key, user or team
- Paid onlySpend analytics are in the hosted app
- CachingExact or semantic response caching
- PartlyIn-memory exact cache; semantic cache is paid
- GuardrailsPII redaction, content filters or prompt-injection checks
- YesBuilt-in checks and partner plugins; PII redaction is listed under enterprise
- MCP gatewayExposes or governs MCP servers and tools
- Paid onlyHosted and enterprise; open source code only on the unreleased 2.0.0 branch
At a glance
- Type
- LLM gateway
- Stack
- TypeScript / Node.js
- Databases
- Redis
- SSO
- Paid only
- Roles and permissions
- Paid only
- Audit logs
- Paid only
Free vs paid editions
What you get when you self-host, and what needs a paid plan.
The open source gateway is MIT licensed and covers the unified API, retries, fallbacks, load balancing, conditional routing, guardrail checks, an in-memory cache and request and token rate limits per provider. Portkey's hosted plans add stored logs, spend analytics, semantic caching and RBAC from $49 a month, and the enterprise version adds SSO, audit logs, budgets and private deployment. Palo Alto Networks bought Portkey in May 2026, and the hosted product is now sold as Prisma AIRS AI Gateway.
Self-hosting Portkey AI Gateway
Portkey gives no minimum CPU or RAM, and the quick start only needs Node.js and npm, or Docker.
New cloud server to ready
8 s
Memory in use at idle
36 MB of 4 GB
Median request
2.8 ms
Requests/s, 32 clients
615
Peak memory
111 MB
Version tested
1.15.2 · Oct 21, 2026
- One container, ready in 8 seconds and 36 MB at idle. Against the mock it added 2.8 ms per request and handled 615 requests a second with 32 clients.
- There are no virtual keys or stored settings in the open source gateway. Each client sends the provider key and the routing config in a header, so every app needs the real provider key.
- Run as documented, the console at /public and the live log stream at /log/stream needed no login, and the stream showed each request's prompt and the provider key from the config header. Setting NODE_ENV=production turned both off. A fix was merged in May 2026 but has not been released.
- The fallback worked, with x-portkey-last-used-option-index pointing at the second target.
docker run -d -p 127.0.0.1:8787:8787 portkeyai/gateway:1.15.2
# Add -e NODE_ENV=production: without it, /public and /log/stream need no login.
# The gateway keeps no state: each request carries its routing config.
curl http://127.0.0.1:8787/v1/chat/completions \
-H 'Content-Type: application/json' \
-H 'x-portkey-config: {"strategy": {"mode": "fallback"}, "targets": [
{"provider": "openrouter", "api_key": "sk-or-…", "override_params": {"model": "anthropic/claude-model-that-does-not-exist"}},
{"provider": "openrouter", "api_key": "sk-or-…", "override_params": {"model": "anthropic/claude-haiku-5.5"}}]}' \
-d '{"messages": [{"role": "user", "content": "In one sentence, what does an LLM gateway do?"}], "max_tokens": 60}'x-portkey-last-used-option-index: config.targets[1]
x-portkey-provider: openrouter
{
"model": "anthropic/claude-haiku-5.5",
"choices": [
{
"message": {
"role": "assistant",
"content": "An LLM gateway is a proxy layer that sits between applications and one or more large language model providers, centralizing access to them so teams can route requests, manage API keys, enforce rate limits and access"
},
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 60,
"total_tokens": 85,
"cost": 3.25e-05
}
}What you can do with Portkey AI Gateway
Provider failover for a production app
Send all LLM calls through one endpoint and fall back to a second provider when the first returns errors or rate limits.
Edge gateway on Cloudflare Workers
Deploy the gateway as a Worker so requests are routed and retried close to users without running servers.
Block unwanted replies
Add output guardrails that reject responses containing banned words or failing a JSON schema before they reach users.
Portkey AI Gateway alternatives
BifrostGo gateway that puts many LLM providers behind one OpenAI-compatible API, with failover, virtual keys, budgets, semantic caching and an MCP gateway
LiteLLMPython SDK and proxy server that puts 100+ LLM providers behind one OpenAI-format API, with virtual keys, budgets and spend tracking
agentgatewayRust proxy for LLM, MCP and agent-to-agent traffic, with per-key budgets, guardrails and a built-in UI
Apache APISIXAPI gateway on NGINX and etcd with AI plugins for LLM proxying, fallbacks, token limits, caching and guardrails
New APISelf-hosted AI gateway with a web console for routing, quotas, billing and usage logs, built on One API
HigressAPI gateway built on Istio and Envoy, with Wasm plugins for LLM proxying, token quotas, semantic caching and MCP hosting