Geekflare

Best Open Source LLM Gateways in 2026

I looked at 23 open source LLM gateways, also called AI gateways, and ranked the ones I'd put between a company's apps and its model providers. Then I ran LiteLLM, Bifrost, agentgateway, Apache APISIX and Portkey on one cloud server, measured how much each adds to every request, and sent real calls through each to OpenRouter, including one to a model that doesn't exist, to see whether they fall back.

Data refreshedMethodologyHow we evaluate
Founder of Geekflare

Chandan Kumar is a founder of Geekflare. He is a technology enthusiast and entrepreneur who loves to help businesses and people around the world. Chandan has worked at BNP Paribas, Citibank, Deutsche Bank, Motorola, and HP. He’s got an in-depth understanding of business software and IT infrastructure.

What I found

  • Against the same instant provider, agentgateway and Apache APISIX added about half a millisecond per request and Bifrost about 1 ms, while LiteLLM added 18 ms and handled 55 requests a second on one worker, against 4,300 for APISIX.
  • All five fell back from a model that doesn't exist to Claude Haiku, but not in the same way. APISIX needed 400 added to its fallback statuses, and agentgateway only switched from the next request on, leaving the first one to fail.
  • LiteLLM put a credential stealer on PyPI and Docker on the day of my test, its second compromised release this year. I checked the image I pulled against its notice; pin exact versions and check digests if you run it.
  • Portkey's released gateway, run as its docs show, put every prompt and the provider key from each request on a log stream anyone could open. Bifrost was the opposite, refusing requests without a virtual key and keeping its dashboard locked until an admin is set up.

Tested on DigitalOcean Droplet with Ubuntu 24.04 server with 2 vCPU and 4 GB of RAM, using Docker. “Ready in” runs from an empty server to the first working screen or result, image downloads included. “Memory at idle” is what the software and its database use together with no traffic. For overhead, each gateway sat in front of the same stand-in provider, an NGINX server that answers every chat completion instantly, so the numbers show the gateway and not a model. The median request is 1,000 requests one at a time, against 0.17 ms with no gateway, and requests per second is 32 clients for 20 seconds. Real calls went through each gateway to Claude Haiku 5.5 on OpenRouter.

SoftwareVersionReady inMemory at idleMedian requestRequests/s, 32 clientsPeak memoryWhat happened
Bifrost2.3.219 s180 MB of 4 GB1.2 ms970254 MBAbout 1 ms added per request and 970 requests a second. Out of the box it refuses requests without a virtual key and blocks providers on private IPs, so plan a little setup.
LiteLLM1.106.21 min 1 s642 MB of 4 GB18.1 ms55772 MBThe slowest and heaviest here: 18 ms added per request and 55 requests a second on one worker, which is what its production guide recommends per pod. Fallback worked and every response carried its cost.
agentgateway1.6.04 s8 MB of 4 GB0.46 ms3,33676 MBReady in 4 seconds, 8 MB at idle and 0.5 ms added per request. Failover worked from the next request on; the first request to the broken model still returned its error.
Apache APISIX3.19.09 s68 MB of 4 GB0.49 ms4,30291 MBThe fastest under load, at 4,300 requests a second, with 0.5 ms added per request. The fallback needed 400 added to fallback_http_statuses, since by default it only falls back on 429 and 5xx.
Portkey AI Gateway1.15.28 s36 MB of 4 GB2.8 ms615111 MBLight and quick to start, with 2.8 ms added per request. Run as documented, its open log stream showed every prompt and the provider key sent in the config header.

8 Open Source LLM Gateways compared

Licence, deployment options and the vendor's hosted plan side by side. Several of these are general API gateways with AI plugins, which matters if you already run one.

LicenceTypeDockerSSOManaged cloud
1
Bifrost
Go
Apache-2.0
Open core
LLM gateway9kOct 2026(Actively maintained)YesPaid onlyNone
2
LiteLLM
Python / Rust
MIT
Open core
Python SDK with a proxy server62kOct 2026(Actively maintained)YesPaid onlyNone
3
agentgateway
Rust
Apache-2.0
Permissive
LLM, MCP and A2A gateway5.5kOct 2026(Actively maintained)YesFree editionNone
4
Apache APISIX
Lua / NGINX
Apache-2.0
Permissive
API gateway with AI plugins17kSep 2026(Actively maintained)YesPaid onlyUsage-based
5
New API
Go / React
AGPL-3.0
Copyleft
LLM gateway50kApr 2026(Actively maintained)YesFree editionNone
6
Higress
Go / Envoy
Apache-2.0
Permissive
API gateway with AI plugins9.6kOct 2026(Actively maintained)YesNoUsage-based
7
Agent Router
Go / Envoy
Apache-2.0
Permissive
Kubernetes AI gateway2.3kOct 2026(Actively maintained)YesNoUsage-based
8
Portkey AI Gateway
TypeScript / Node.js
MIT
Permissive
LLM gateway13kJan 2026(Actively maintained)YesPaid only$49/month
  • Actively maintained
  • Slowing: last release 1 to 2 years ago
  • Stale: no release in over 2 years

What each one can do

The features teams usually put a gateway in for, checked against each project's docs and, for the five I tested, on my server. Paid means the feature is only in a paid edition, a licence key or the vendor's cloud.

SoftwareFallbacksLoad balancingBudgets and limitsCost trackingCachingGuardrailsMCP gateway
BifrostYesYesWeighted keys; adaptive balancing is EnterpriseYesPer virtual key, team and customerYesYesExact and semantic, needs a vector storePaid onlyYesCode Mode and per-key tool filtering included
LiteLLMYesYesYesPer key, user and team; tag and per-model key budgets need EnterpriseYesYesExact and semantic, Redis, Qdrant, Valkey, S3 or GCSYesPresidio and custom guardrails free; some integrations need EnterpriseYes
agentgatewayYesPriority groups with health-based evictionYesYesPer-key dollar or token budgets; limits across replicas are EnterpriseYesNeeds SQLite or PostgreSQLNoYesYes
Apache APISIXYesYesWeighted, consistent hash or semanticYesToken limits per route, instance or consumer; no dollar capsPartlyToken counts per consumer and model, not pricesYesExact and semantic in Redis, since 3.18YesRegex, AWS Comprehend, Aliyun and Lakera GuardPartlyopenapi-to-mcp serves REST APIs as MCP tools; mcp-bridge is deprecated
New APIYesRetries on other channels and groupsYesChannel weights and multi-key pollingYesQuotas per key and user, rate limits per groupYesNoBills provider prompt caching onlyPartlySensitive word list, off by defaultNo
HigressYesYesAcross keys, and LLM-aware policies via ai-load-balancerYesToken quotas and limits per consumer; needs RedisPartlyToken counts per consumer and model, no dollar amountsYesRedis exact match or vector-database semantic cacheYesLocal data masking; moderation via Alibaba CloudYes
Agent RouterYesYesWeighted backends and InferencePoolYesToken limits per user, tenant or model; needs RedisPartlyToken counts in metrics and logs, no dollar amountsNoPasses provider prompt-cache hints onlyNoYes
Portkey AI GatewayYesYesPartlyRequest and token limits per provider in conf.json; spend budgets per key are paidPaid onlySpend analytics are in the hosted appPartlyIn-memory exact cache; semantic cache is paidYesBuilt-in checks and partner plugins; PII redaction is listed under enterprisePaid onlyHosted and enterprise; open source code only on the unreleased 2.0.0 branch
Ask Sparkian

Which LLM gateway fits your team?

Describe your apps, the providers you use and what you need to control, such as budgets or failover, and AI will shortlist the best fits from these picks.

Or try:

LLM gateways I'd recommend

Ranked on what a team needs from a gateway it puts in front of every model call: failover that works, budgets and cost tracking, how little it adds to each request, safe defaults, and which of those stay free.

  1. #1Best all-round LLM gatewayTested by Geekflare
    Full reviewWebsite GitHub

    Bifrost

    Go gateway that puts many LLM providers behind one OpenAI-compatible API, with failover, virtual keys, budgets, semantic caching and an MCP gateway

    Fast and light, with fallbacks, virtual keys, budgets, caching and an MCP gateway in the free version, and safe defaults. SSO, roles, audit logs, guardrails and clustering are in Bifrost Enterprise.

    Bifrost 2.3.2 asking for a setup token before the dashboard can be used

    Bifrost 2.3.2 asking for a setup token before the dashboard can be used

    Tested Oct 21, 2026, version 2.3.2
    Ready in
    19 s
    Memory in use at idle
    180 MB of 4 GB
    Median request
    1.2 ms
    Requests/s, 32 clients
    970
    Peak memory
    254 MB

    About 1 ms added per request and 970 requests a second. Out of the box it refuses requests without a virtual key and blocks providers on private IPs, so plan a little setup.

    Commands, notes and screenshots

    Best for

    Teams that want a fast single-node gateway with budgets and an MCP gateway

    Not for

    Companies that need SSO, audit logs or high availability without a paid licence

    • Added about 1 ms per request in our test, against 18 ms for LiteLLM
    • Virtual keys, budgets and the MCP gateway are in the free version
    • Starts with one command and is configured from a web dashboard
    • Built-in importer for an existing LiteLLM proxy setup
    • Guardrails, SSO, roles and audit logs are Enterprise only
    • Several open source nodes cannot share state, so high availability needs Enterprise clustering
    • The documented Docker command fails unless the data folder belongs to UID 1000
    Licence
    Apache-2.0 (open core)
    Stack
    Go
    Databases
    SQLite, PostgreSQL
    GitHub
    9k stars · 201 contributors
    Latest release
    2.3.2 · Oct 20, 2026
    SSO
    Paid only
    Audit logs
    Paid only
    What people say
  2. #2Widest provider coverage and team budgetsTested by Geekflare
    Full reviewWebsite GitHub

    LiteLLM

    Python SDK and proxy server that puts 100+ LLM providers behind one OpenAI-format API, with virtual keys, budgets and spend tracking

    The most complete free proxy for keys, team budgets, spend tracking and fallbacks across almost any provider. It is the heaviest and slowest here, SSO beyond 5 users needs a licence, and two of its 2026 releases were compromised.

    Tested Oct 21, 2026, version 1.106.2
    Ready in
    1 min 1 s
    Memory in use at idle
    642 MB of 4 GB
    Median request
    18.1 ms
    Requests/s, 32 clients
    55
    Peak memory
    772 MB

    The slowest and heaviest here: 18 ms added per request and 55 requests a second on one worker, which is what its production guide recommends per pod. Fallback worked and every response carried its cost.

    Commands, notes and screenshots

    Best for

    Platform teams that need one API and per-team budgets across many providers

    Not for

    Teams that cannot pin versions and verify release artifacts before upgrading

    • One OpenAI-format API for 100+ providers, including Bedrock, Vertex AI and Azure
    • Virtual keys, team budgets and spend tracking in the free version
    • Works as an in-process Python library or a central proxy
    • Exact and semantic caching with several backends
    • Credential-stealing releases reached PyPI in March and PyPI and Docker in October 2026
    • SSO beyond 5 users, audit logs and team admin roles need an Enterprise licence
    • The Python proxy needs 4 GB of RAM per worker and the Rust core is still partial
    Licence
    MIT (open core)
    Stack
    Python / Rust
    Databases
    PostgreSQL, Redis
    GitHub
    62k stars · 1.9k contributors
    Latest release
    v1.106.2 · Oct 21, 2026
    SSO
    Paid only
    Audit logs
    Paid only
    What people say
  3. #3Best for agents and MCPTested by Geekflare
    Full reviewWebsite GitHub

    agentgateway

    Rust proxy for LLM, MCP and agent-to-agent traffic, with per-key budgets, guardrails and a built-in UI

    One Rust proxy for LLM calls, MCP servers and agent-to-agent traffic, under the Linux Foundation, with budgets and a cost dashboard free. The lightest gateway I tested; it has no response cache.

    Tested Oct 21, 2026, version 1.6.0
    Ready in
    4 s
    Memory in use at idle
    8 MB of 4 GB
    Median request
    0.46 ms
    Requests/s, 32 clients
    3,336
    Peak memory
    76 MB

    Ready in 4 seconds, 8 MB at idle and 0.5 ms added per request. Failover worked from the next request on; the first request to the broken model still returned its error.

    Commands, notes and screenshots

    Best for

    Teams that want one gateway for LLM calls, MCP servers and agents

    Not for

    Teams that need response caching in the gateway

    • Handles LLM, MCP and A2A traffic in one proxy
    • Runs as a single binary with no Kubernetes needed
    • Per-key dollar budgets and a cost dashboard in the free version
    • Several guardrail types that can run in sequence
    • Rate limits are local to each instance unless you pay for Solo Enterprise
    • No response caching
    • Fast release pace means frequent upgrades
    • In the simple LLM config, failover helps the next request while the failing one still errors
    Licence
    Apache-2.0 (permissive)
    Stack
    Rust
    Databases
    SQLite, PostgreSQL
    GitHub
    5.5k stars · 280 contributors
    Latest release
    v1.6.0 · Oct 2, 2026
    SSO
    Free edition
    Audit logs
    Free edition
  4. #4Best free API gateway with LLM routingTested by Geekflare
    Full reviewWebsite GitHub

    Apache APISIX

    API gateway on NGINX and etcd with AI plugins for LLM proxying, fallbacks, token limits, caching and guardrails

    Fallbacks, token limits, a semantic cache and guardrail plugins with no licence key, and the highest throughput in my test. You configure each route by hand, and it counts tokens rather than dollars.

    Tested Oct 21, 2026, version 3.19.0
    Ready in
    9 s
    Memory in use at idle
    68 MB of 4 GB
    Median request
    0.49 ms
    Requests/s, 32 clients
    4,302
    Peak memory
    91 MB

    The fastest under load, at 4,300 requests a second, with 0.5 ms added per request. The fallback needed 400 added to fallback_http_statuses, since by default it only falls back on 429 and 5xx.

    Commands, notes and screenshots

    Best for

    Teams that want a free API gateway with real LLM routing, limits and caching

    Not for

    Teams that want a ready-made LLM portal with keys, budgets and spend reports

    • Fallbacks, health checks and weighted or semantic load balancing for free
    • Token rate limits per route, model instance or consumer
    • Exact and semantic response cache in Redis
    • A new release roughly every two months
    • Tracks tokens but not spend in dollars
    • Shared admin keys, with per-user logins and audit logs only in API7 Enterprise
    • Each LLM route is configured by hand as plugin JSON or YAML
    Licence
    Apache-2.0 (permissive)
    Stack
    Lua / NGINX
    Databases
    etcd, Redis
    GitHub
    17k stars · 522 contributors
    Latest release
    3.19.0 · Sep 28, 2026
    Managed cloud
    API7 Cloud, Usage-based
    SSO
    Paid only
    Audit logs
    Paid only
    What people say
  5. #5Best for sharing provider keys with quotas
    Website GitHub

    New API

    Self-hosted AI gateway with a web console for routing, quotas, billing and usage logs, built on One API

    A web console for many users and keys with per-call costs, quotas and format conversion between OpenAI, Anthropic and Gemini. Much of it is built for reselling API credit, and it has shipped only release candidates since April 2026.

    Best for

    Teams that share provider keys across many users and want quotas and per-call costs

    Not for

    Teams that need response caching, PII guardrails or an MCP gateway

    • Converts between OpenAI, Anthropic and Gemini request formats
    • Channel priorities, weights, retries and multi-key polling
    • OIDC login, admin permissions and audit logs in the free version
    • Runs on SQLite in one container or on PostgreSQL and Redis across nodes
    • No response caching or MCP gateway
    • No stable release since v0.13.2 in April 2026, only v1 release candidates
    • Top-ups, redemption codes and invitations clutter internal use
    Licence
    AGPL-3.0 (copyleft)
    Stack
    Go / React
    Databases
    SQLite, MySQL, PostgreSQL, Redis, ClickHouse
    GitHub
    50k stars · 347 contributors
    Latest release
    0.13.2 · Apr 27, 2026
    SSO
    Free edition
    Audit logs
    Free edition
  6. #6Best Kubernetes gateway with a console
    Website GitHub

    Higress

    API gateway built on Istio and Envoy, with Wasm plugins for LLM proxying, token quotas, semantic caching and MCP hosting

    An Envoy-based gateway for APIs and LLM traffic with caching, token quotas, data masking and MCP hosting, managed from a free web console. Production means Kubernetes or Alibaba Cloud.

    Best for

    Teams that want one Envoy gateway for APIs, microservices and LLM traffic

    Not for

    Teams that want a light LLM proxy without Kubernetes

    • Exact and semantic response caching
    • Web console for providers, AI routes and consumers
    • Token quotas and rate limits per consumer
    • Also works as an ingress controller that reads NGINX Ingress annotations
    • Docker and standalone modes are meant for testing, not production
    • Content moderation plugin needs a paid Alibaba Cloud service
    • Some docs and the docs index are only in Chinese
    Licence
    Apache-2.0 (permissive)
    Stack
    Go / Envoy
    Databases
    Redis, Nacos
    GitHub
    9.6k stars · 214 contributors
    Latest release
    v2.2.5 · Oct 4, 2026
    Managed cloud
    Alibaba Cloud AI Gateway, Usage-based
    SSO
    No
    Audit logs
    Free edition
  7. #7Best for teams on Envoy Gateway
    Website GitHub

    Agent Router

    Kubernetes control plane for LLM and MCP traffic built on Envoy Gateway, formerly Envoy AI Gateway

    Token quotas, provider fallback and MCP routing as Kubernetes resources, formerly Envoy AI Gateway. It needs Kubernetes and Envoy Gateway, and has no UI or dollar cost tracking.

    Best for

    Platform teams that already run Envoy Gateway on Kubernetes

    Not for

    Small teams that want a UI and spend dashboards without Kubernetes

    • Routes to self-hosted models with the Gateway API Inference Extension
    • Token quotas per model and per tenant, with a shadow mode for testing
    • MCP gateway with OAuth and tool filtering
    • aigw run tests the same configuration on a laptop
    • Production use needs Kubernetes and Envoy Gateway
    • Tracks tokens, not dollar spend, and has no UI
    • Several MCP authorisation flaws were fixed in v1.2.0, with breaking changes
    Licence
    Apache-2.0 (permissive)
    Stack
    Go / Envoy
    Databases
    Redis
    GitHub
    2.3k stars · 165 contributors
    Latest release
    v1.2.0 · Oct 7, 2026
    Managed cloud
    Tetrate Agent Router Service, Usage-based
    SSO
    No
    Audit logs
    Free edition
  8. #8Light stateless gateway, now stalledTested by Geekflare
    Full reviewWebsite GitHub

    Portkey AI Gateway

    TypeScript LLM gateway with one OpenAI-style API, retries, fallbacks, load balancing and guardrail hooks

    A small service with JSON configs for fallbacks, retries and guardrails that runs anywhere, including Cloudflare Workers. No release since January 2026, and run without NODE_ENV=production it exposes prompts and keys in its log stream.

    Live request logs in the Portkey console, open to anyone who can reach the port

    Live request logs in the Portkey console, open to anyone who can reach the port

    Tested Oct 21, 2026, version 1.15.2
    Ready in
    8 s
    Memory in use at idle
    36 MB of 4 GB
    Median request
    2.8 ms
    Requests/s, 32 clients
    615
    Peak memory
    111 MB

    Light and quick to start, with 2.8 ms added per request. Run as documented, its open log stream showed every prompt and the provider key sent in the config header.

    Commands, notes and screenshots

    Best for

    Teams that want a light routing and guardrail proxy they can run anywhere, including at the edge

    Not for

    Teams that need budgets, spend tracking or admin logins in the free version

    • Retries, fallbacks, load balancing and conditional routing from one JSON config
    • Runs on Node.js, Bun, Docker or Cloudflare Workers
    • Built-in guardrail checks plus partner plugins
    • MIT licence with no licence-key features
    • No release since January 2026 and no merges since May 2026
    • Semantic caching, spend analytics, RBAC and SSO are in paid plans
    • The released 1.15.2 shows prompts and provider keys on an open log stream unless NODE_ENV is production
    Licence
    MIT (permissive)
    Stack
    TypeScript / Node.js
    Databases
    Redis
    GitHub
    13k stars · 123 contributors
    Latest release
    v1.15.2 · Jan 12, 2026
    Managed cloud
    Portkey, $49/month
    SSO
    Paid only
    Audit logs
    Paid only

More open source LLM gateways

An API gateway whose AI features are mostly paid, a gateway built into MLflow, and two newer or narrower projects.

SoftwareLicenceTypeGitHub starsLatest release
Kong AI Gateway
Kong Gateway with AI plugins that proxy several LLM providers behind one API, with prompt guards and templates
Apache-2.0
Open core
API gateway with AI plugins44k(Actively maintained)Oct 2026
MLflow AI Gateway
LLM gateway built into the MLflow tracking server, with fallbacks, traffic splitting, budgets, guardrails and request tracing
Apache-2.0
Permissive
LLM gateway inside an MLOps platform28k(Actively maintained)Oct 2026
LLM Gateway
OpenAI-compatible LLM gateway with provider routing, usage and cost analytics, and a hosted version
AGPL-3.0 / LLM Gateway Enterprise
Open core
LLM gateway1.7k(Actively maintained)Oct 2026
Plano
Envoy-based proxy for agent apps that routes between LLMs and agents and runs guardrail filters, formerly Arch Gateway
Apache-2.0
Permissive
AI proxy for agents7.1k(Actively maintained)Sep 2026
  • Actively maintained
  • Slowing: last release 1 to 2 years ago
  • Stale: no release in over 2 years

Not included, and why

Gateways that stopped, moved or changed licence, tools that do a different job, and the hosted services people compare these with.

  • Helicone: Mintlify bought it in March 2026 and put it in maintenance mode. The self-hosted version no longer proxies LLM traffic, and its separate Rust gateway hasn't been updated since November 2025.
  • One API: No commits since February 2025. New API, which is built on it, is the active continuation.
  • kgateway: Removed its AI features in 2026 and points to agentgateway instead.
  • vLLM Semantic Router: A routing layer that runs behind a gateway such as agentgateway, without budgets, cost tracking or failover of its own.
  • vllora: Formerly LangDB AI Gateway, it is now a debugging tool for agents under the Elastic License, which isn't open source.
  • RouteLLM, BricksLLM and Glide: No commits since August 2024, January 2025 and August 2024.
  • 9router: Routes personal coding subscriptions such as Claude Code and Codex logins, which some providers' terms don't allow, rather than a company's API traffic.
  • OpenRouter, Cloudflare AI Gateway, Vercel AI Gateway and TrueFoundry: Hosted services, not open source.

How to choose

Decide what the gateway is for

If you need keys, budgets and spend reports for many teams, look at LiteLLM, Bifrost or New API. If you want one proxy for model calls, MCP tools and agents, agentgateway. If you already run an API gateway, APISIX, Higress or Kong can add LLM routing to it.

Measure the cost of the extra hop

In my test the fastest gateways added under 1 ms per request and LiteLLM added 18 ms. Next to a model call of a second or more that matters little, but for high-volume or streaming traffic, or on small servers, it decides how many instances you run.

Check what's free before you plan

Most open source gateways keep SSO, audit logs and roles for a paid edition, and some keep more. Kong's free edition has no fallbacks, Bifrost's has no guardrails, and LiteLLM needs a licence for SSO beyond 5 users.

Treat it as security infrastructure

A gateway holds every provider key and sees every prompt. Pin exact versions, check image digests, bind admin ports to private networks, and turn on authentication, since several gateways, including agentgateway and APISIX, accept requests without a key by default.

How we evaluate

No vendor influences these pages. A vendor can pay for a Featured placement, labelled Sponsored, which never changes a rank or a review.

What qualifies
Self-hostable gateways and API gateways with LLM routing, under an OSI open source licence or a clearly labelled open-core or source-available one, with a release or commit in the last 12 months and 1,000+ GitHub stars. Hosted-only services, client libraries without a proxy, routers that sit behind a gateway, and tools for sharing personal AI subscriptions are not considered.
Where we look
GitHub, the pages that rank for each category, so both established and newer projects are considered.
How we rank
Fit for business and technical teams: maintenance activity, self-hosting effort, licence terms, and which features the paid editions lock. Payment never changes a ranking.
Where the data comes from
Stars, releases, contributors and licences come from GitHub. Editions, pricing and requirements come from each software's own docs.
Hands-on testing
We install several of the picks in each category on a cloud Ubuntu 24.04 server (2 vCPU, 4 GB RAM) with Docker, and record install time and idle memory.
Questions

Frequently asked questions

An LLM gateway, or AI gateway, is a proxy between your applications and model providers. Apps call one OpenAI-style API, and the gateway handles provider keys, routing, fallbacks, budgets, cost tracking, caching and logging.

For most teams, Bifrost, because it is fast, light and keeps fallbacks, budgets, caching and an MCP gateway free. For the widest provider coverage and team budgets, LiteLLM, and for agents and MCP, agentgateway.

Bifrost is the closest like-for-like replacement, with virtual keys, budgets and a dashboard, and it can import a LiteLLM setup. agentgateway suits agent and MCP traffic, and Apache APISIX suits teams that already run an API gateway.

OpenRouter is a hosted service that resells access to many models with one bill. LiteLLM is software you run yourself with your own provider keys, so you control keys, budgets and logs but run the servers. They also work together, since LiteLLM can route to OpenRouter.

Its code is widely used, but releases were compromised twice in 2026, in March on PyPI and on October 21 on PyPI and Docker. Pin exact versions, verify image digests against LiteLLM's notices, and rotate keys if you ran an affected build.