Geekflare
PermissiveApache-2.0Tested by Geekflare

Apache APISIX review

API gateway on NGINX and etcd with AI plugins for LLM proxying, fallbacks, token limits, caching and guardrails

Website GitHub Docs
Data refreshedMethodologyHow we tested
Founder of Geekflare

Chandan Kumar is a founder of Geekflare. He is a technology enthusiast and entrepreneur who loves to help businesses and people around the world. Chandan has worked at BNP Paribas, Citibank, Deutsche Bank, Motorola, and HP. He’s got an in-depth understanding of business software and IT infrastructure.

Verdict

APISIX puts fallbacks, weighted and semantic load balancing, token limits per consumer, guardrail plugins and a Redis cache with a semantic layer in the free Apache 2.0 edition, with no licence key. The catch is that it is a general gateway you configure route by route through an Admin API or YAML, there is no spend tracking in dollars, and team logins and audit logs need API7 Enterprise.

Best for

Teams that want a free API gateway with real LLM routing, limits and caching

Not for

Teams that want a ready-made LLM portal with keys, budgets and spend reports

Pros

  • Fallbacks, health checks and weighted or semantic load balancing for free
  • Token rate limits per route, model instance or consumer
  • Exact and semantic response cache in Redis
  • A new release roughly every two months

Cons

  • Tracks tokens but not spend in dollars
  • Shared admin keys, with per-user logins and audit logs only in API7 Enterprise
  • Each LLM route is configured by hand as plugin JSON or YAML
What people say
Ask Sparkian

Is Apache APISIX right for your project?

Describe what you need and AI will check the fit against this review.

Or try:

Apache APISIX features and capabilities

One route for many providers

The ai-proxy plugin converts requests for OpenAI, Anthropic, Gemini, Vertex AI, Bedrock, Azure, DeepSeek and other OpenAI-compatible APIs, including embedding models, so clients talk to one endpoint.

Fallbacks and semantic routing

ai-proxy-multi retries another instance on rate limits, server errors or statuses you list, and can pick a model by how close the prompt is to example prompts for each instance.

Token limits that match your plans

ai-rate-limiting caps prompt, completion or total tokens per time window for each route, model instance or consumer, with counters shared across nodes through Redis.

Two-layer response cache

ai-cache replays answers for identical prompts from Redis and can also serve close matches by embedding similarity, which cuts token spend for FAQ and document bots.

Guardrails and MCP tools

Prompts can be checked with regex rules, AWS Comprehend, Aliyun or Lakera Guard, and the openapi-to-mcp plugin serves an existing REST API to agents as MCP tools.

FallbacksSwitches to another model or provider when a call fails
Yes
Load balancingSpreads traffic across keys, deployments or providers
YesWeighted, consistent hash or semantic
Budgets and limitsSpending caps or rate limits per key, user or team
YesToken limits per route, instance or consumer; no dollar caps
Cost trackingRecords spend per request and per key, user or team
PartlyToken counts per consumer and model, not prices
CachingExact or semantic response caching
YesExact and semantic in Redis, since 3.18
GuardrailsPII redaction, content filters or prompt-injection checks
YesRegex, AWS Comprehend, Aliyun and Lakera Guard
MCP gatewayExposes or governs MCP servers and tools
Partlyopenapi-to-mcp serves REST APIs as MCP tools; mcp-bridge is deprecated

At a glance

Type
API gateway with AI plugins
Stack
Lua / NGINX
Databases
etcd, Redis
SSO
Paid only
Roles and permissions
Free edition
Audit logs
Paid only

Free vs paid editions

What you get when you self-host, and what needs a paid plan.

Every APISIX plugin is free under Apache 2.0, including AI proxying with fallbacks and load balancing, token rate limits, exact and semantic caching and the content moderation plugins. API7 Enterprise, from the company behind APISIX, adds per-user dashboard accounts with RBAC, SSO, configuration audit logs, multi-cluster management and a developer portal, licensed by CPU core with a custom quote. API7 Cloud is pay-as-you-go, at $250 a month per gateway group plus $10 per service and $2 per million calls.

Self-hosting Apache APISIX

APISIX gives no minimum CPU or RAM, and needs an etcd cluster unless you run it in standalone mode from a YAML file.

New cloud server to ready

9 s

Memory in use at idle

68 MB of 4 GB

Median request

0.49 ms

Requests/s, 32 clients

4,302

Peak memory

91 MB

Version tested

3.19.0 · Oct 21, 2026

  • We ran it in standalone mode from a YAML file, without etcd. The container was ready in 9 seconds and used 68 MB at idle.
  • Against the mock it added about 0.5 ms per request and handled 4,302 requests a second with 32 clients, the most of any gateway we tested, peaking at 91 MB.
  • Our first real calls failed with 401: APISIX only resolves $env:// secrets when they are the whole value, not inside "Bearer $env://…". Passing the full header value through one variable fixed it.
  • Each model is a route you write by hand, and there is no login on the routes unless you add an auth plugin.
Routes we ran (standalone mode, apisix.yaml)
routes:
  - id: mock                                 # the stand-in provider for the overhead test
    uri: /mock/v1/chat/completions
    plugins:
      ai-proxy:
        provider: openai-compatible
        options: { model: mock-model }
        override: { endpoint: "http://mock/v1/chat/completions" }
  - id: broken
    uri: /broken/v1/chat/completions
    plugins:
      ai-proxy-multi:
        fallback_strategy: ["http_429", "http_5xx"]
        fallback_http_statuses: [400]        # the broken model returns 400
        max_retries: 1
        instances:
          - name: broken-primary
            provider: openrouter
            priority: 1
            weight: 1
            auth: { header: { Authorization: "$env://OPENROUTER_BEARER" } }
            options: { model: anthropic/claude-model-that-does-not-exist }
          - name: haiku-backup
            provider: openrouter
            priority: 0
            weight: 1
            auth: { header: { Authorization: "$env://OPENROUTER_BEARER" } }
            options: { model: anthropic/claude-haiku-5.5 }
#END
OutputPassed3.19.0 · fell back to Claude Haiku 5.5 · 1.2 s
Request: POST /broken/v1/chat/completions, a route whose first instance asks for a model that doesn't exist; asked "In one sentence, what does an LLM gateway do?"
{
  "model": "anthropic/claude-haiku-5.5",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "An LLM gateway is a proxy layer that sits between applications and one or more large language model providers, centralizing access to them and handling concerns like routing, authentication, rate limiting, cost t"
      },
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 60,
    "total_tokens": 85,
    "cost": 3.25e-05
  }
}

What you can do with Apache APISIX

Shared LLM endpoint with limits

Give each internal team a consumer with its own token quota and route all of them through one set of provider keys.

Failover between providers

Send traffic to a primary model and switch to a second provider when it returns 429 or 5xx errors.

Cheaper support bots

Cache answers to repeated or similar questions in Redis so a help bot calls the model less often.

Apache APISIX alternatives

BifrostGo gateway that puts many LLM providers behind one OpenAI-compatible API, with failover, virtual keys, budgets, semantic caching and an MCP gatewayLiteLLMPython SDK and proxy server that puts 100+ LLM providers behind one OpenAI-format API, with virtual keys, budgets and spend trackingagentgatewayRust proxy for LLM, MCP and agent-to-agent traffic, with per-key budgets, guardrails and a built-in UINew APISelf-hosted AI gateway with a web console for routing, quotas, billing and usage logs, built on One APIHigressAPI gateway built on Istio and Envoy, with Wasm plugins for LLM proxying, token quotas, semantic caching and MCP hostingAgent RouterKubernetes control plane for LLM and MCP traffic built on Envoy Gateway, formerly Envoy AI Gateway