Geekflare
Open coreApache-2.0Tested by Geekflare

Bifrost review

Go gateway that puts many LLM providers behind one OpenAI-compatible API, with failover, virtual keys, budgets, semantic caching and an MCP gateway

Website GitHub Docs
Data refreshedMethodologyHow we tested
Founder of Geekflare

Chandan Kumar is a founder of Geekflare. He is a technology enthusiast and entrepreneur who loves to help businesses and people around the world. Chandan has worked at BNP Paribas, Citibank, Deutsche Bank, Motorola, and HP. He’s got an in-depth understanding of business software and IT infrastructure.

Verdict

Bifrost is a single Go binary with a web dashboard that added about 1 ms per request in our test, and the free version already covers failover, budgets per virtual key, caching and an MCP gateway. The catch is that guardrails, SSO, roles, audit logs and clustering are Enterprise only, and the docs say several open source nodes cannot share one Postgres database.

Best for

Teams that want a fast single-node gateway with budgets and an MCP gateway

Not for

Companies that need SSO, audit logs or high availability without a paid licence

Pros

  • Added about 1 ms per request in our test, against 18 ms for LiteLLM
  • Virtual keys, budgets and the MCP gateway are in the free version
  • Starts with one command and is configured from a web dashboard
  • Built-in importer for an existing LiteLLM proxy setup

Cons

  • Guardrails, SSO, roles and audit logs are Enterprise only
  • Several open source nodes cannot share state, so high availability needs Enterprise clustering
  • The documented Docker command fails unless the data folder belongs to UID 1000
What people say
Ask Sparkian

Is Bifrost right for your project?

Describe what you need and AI will check the fit against this review.

Or try:

Bifrost features and capabilities

Budgets through virtual keys

Customers, teams and virtual keys each get their own budget and rate limits, so one team or client cannot run up the bill for the rest.

Fallbacks and weighted keys

Calls retry with backoff, switch to another provider when the first fails, and spread across several API keys by weight.

MCP gateway with Code Mode

MCP servers are connected once and filtered per virtual key, and Code Mode lets the model call many tools from one Starlark script instead of loading every tool definition.

Exact and semantic caching

Repeated or similar requests can be answered from a cache in Redis, Valkey, Weaviate, Qdrant or Pinecone, which cuts provider spend on common prompts.

Drop-in SDK endpoints

OpenAI, Anthropic, Bedrock and Google GenAI SDKs work by changing the base URL, and every request is logged with tokens, cost and latency.

FallbacksSwitches to another model or provider when a call fails
Yes
Load balancingSpreads traffic across keys, deployments or providers
YesWeighted keys; adaptive balancing is Enterprise
Budgets and limitsSpending caps or rate limits per key, user or team
YesPer virtual key, team and customer
Cost trackingRecords spend per request and per key, user or team
Yes
CachingExact or semantic response caching
YesExact and semantic, needs a vector store
GuardrailsPII redaction, content filters or prompt-injection checks
Paid only
MCP gatewayExposes or governs MCP servers and tools
YesCode Mode and per-key tool filtering included

At a glance

Type
LLM gateway
Stack
Go
Databases
SQLite, PostgreSQL
Licence terms
The gateway in the repo is Apache 2.0. Guardrails, clustering, adaptive load balancing, SSO, audit logs, roles and secret managers are only in Bifrost Enterprise, a separate paid build.
SSO
Paid only
Roles and permissions
Paid only
Audit logs
Paid only

Free vs paid editions

What you get when you self-host, and what needs a paid plan.

The Apache 2.0 gateway includes fallbacks, weighted load balancing, virtual keys with budgets and rate limits, semantic caching, the MCP gateway with Code Mode and a web dashboard. Bifrost Enterprise, a separate build, adds guardrails, multi-node clustering, adaptive load balancing, SSO, custom roles, audit logs, secret managers and deployment in your VPC or air-gapped. Enterprise has custom pricing and a 14-day free trial, and there is no managed cloud version.

Self-hosting Bifrost

Bifrost gives no minimum requirements for the open source gateway, and its Enterprise sizing guide suggests 4 vCPU and 16 GB of RAM per pod.

New cloud server to ready

19 s

Memory in use at idle

180 MB of 4 GB

Median request

1.2 ms

Requests/s, 32 clients

970

Peak memory

254 MB

Version tested

2.3.2 · Oct 21, 2026

  • The documented docker run with ./data mounted exited at once: the image runs as UID 1000 and couldn't write to a folder owned by root. After chown it was ready in 19 seconds and used 180 MB at idle.
  • Every request without a virtual key got a 401, and the dashboard and management API stayed locked until a setup token is set and an admin created. Good defaults, though the quick start doesn't mention them.
  • It also refused to reach our mock provider on the Docker network until allow_private_network was set, a guard against requests to internal addresses.
  • Against the mock it added about 1 ms per request and handled 970 requests a second with 32 clients, peaking at 254 MB. The fallback request came back from Claude Haiku 5.5 with headers naming the original and the resolved model.
Config we ran (config.json, shown as YAML)
# docker run -p 127.0.0.1:8080:8080 -v ./data:/app/data maximhq/bifrost:v2.3.2
# (chown 1000 ./data first; the image runs as UID 1000)
providers:
  openai:                                  # the stand-in provider for the overhead test
    keys: [{ name: mock, value: none, models: ["*"], weight: 1.0 }]
    network_config: { base_url: "http://mock", allow_private_network: true }
  openrouter:
    keys: [{ name: openrouter, value: env.OPENROUTER_API_KEY, models: ["*"], weight: 1.0 }]
governance:
  virtual_keys:                            # required on every request by default
    - { id: vk-geekflare-test, name: geekflare-test, value: "sk-bf-…", is_active: true, provider_configs: [ … ] }

# The fallback is per request:
# { "model": "openrouter/anthropic/claude-model-that-does-not-exist",
#   "fallbacks": ["openrouter/anthropic/claude-haiku-5.5"], … }
OutputPassed2.3.2 · fell back to Claude Haiku 5.5 · 1.0 s
Request: POST /v1/chat/completions with model openrouter/anthropic/claude-model-that-does-not-exist and fallbacks ["openrouter/anthropic/claude-haiku-5.5"]; asked "In one sentence, what does an LLM gateway do?"
X-Bifrost-Original-Model: anthropic/claude-model-that-does-not-exist
X-Bifrost-Resolved-Model: anthropic/claude-haiku-5.5
X-Bifrost-Routing-Info-Is-Fallback: true

{
  "model": "anthropic/claude-haiku-5.5",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "An LLM gateway is a proxy layer that sits between applications and one or more large language model providers, centralizing access to them and handling concerns such as routing, authentication, rate limiting, cost"
      },
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 60,
    "total_tokens": 85,
    "cost": {
      "total_cost": 3.25e-05
    }
  }
}
Bifrost 2.3.2 asking for a setup token before the dashboard can be used

Bifrost 2.3.2 asking for a setup token before the dashboard can be used

Bifrost 2.3.2 asking for a setup token before the dashboard can be used

What you can do with Bifrost

Shared gateway for a startup

Run one Bifrost instance in front of OpenAI, Anthropic and Bedrock with a virtual key and budget for each team.

Coding agents through one endpoint

Send Claude Code, Codex CLI or Gemini CLI traffic through Bifrost so every model call is logged and capped per key.

Tool access for agents

Connect internal MCP servers once and decide per virtual key which tools each agent may call.

Bifrost alternatives

LiteLLMPython SDK and proxy server that puts 100+ LLM providers behind one OpenAI-format API, with virtual keys, budgets and spend trackingagentgatewayRust proxy for LLM, MCP and agent-to-agent traffic, with per-key budgets, guardrails and a built-in UIApache APISIXAPI gateway on NGINX and etcd with AI plugins for LLM proxying, fallbacks, token limits, caching and guardrailsNew APISelf-hosted AI gateway with a web console for routing, quotas, billing and usage logs, built on One APIHigressAPI gateway built on Istio and Envoy, with Wasm plugins for LLM proxying, token quotas, semantic caching and MCP hostingAgent RouterKubernetes control plane for LLM and MCP traffic built on Envoy Gateway, formerly Envoy AI Gateway