The five-minute definition
An AI Gateway is a service that sits between your application and the LLM providers (OpenAI, Anthropic, Google, Mistral, DeepSeek). Your app talks to the gateway using an OpenAI-compatible API. The gateway handles the rest: authentication, routing, PII scanning, logging, billing, and the provider-specific quirks that eat engineering time.
If you’ve used LiteLLM, you’ve used the routing layer of one. A full AI Gateway bundles the surrounding controls — the ones a team needs in production, not just in a notebook.
The problem with direct provider calls
Most AI features start with a direct call to api.openai.com. It works. It ships. Then reality arrives:
- Every service that needs the LLM reimplements auth. API keys end up in environment variables, in config files, sometimes in source control. Rotation is a fire drill.
- Every service reimplements retry, backoff, and streaming. Provider 429s need jittered exponential backoff. Streaming SSE needs careful buffering. Get either wrong and the feature degrades silently.
- PII lands in provider logs. A user pastes a support ticket into a chat. The ticket has an email, a phone number, sometimes an API key. The prompt goes to the provider verbatim. Now that data lives wherever the provider lives.
- Cost is invisible until the invoice arrives. No per-feature budget. No per-tenant attribution. No way to tell which feature burned 80% of the month’s spend until finance asks.
- Switching providers is a rewrite. OpenAI’s API shape is not Anthropic’s is not Google’s. A model change ripples through every callsite.
None of these are hard individually. Together, they’re a platform team’s backlog.
What an AI Gateway gives you
A gateway collapses those five problems into one integration:
- One auth boundary. Your services present a JWT or API key to the gateway. The gateway holds the provider keys, rotates them, and scopes which service can call which model.
- One routing policy.
"model": "auto"resolves per request based on feature tag, budget, and PII policy. Switch providers in one config change, not a code change. - Pre-call PII scanning. Seventeen categories — API keys, JWTs, PEM private keys, emails, phones, credit cards, JP mynumber, US SSN, connection strings — scanned before the request leaves your boundary. Critical findings block the call.
- Structured logging. Every interaction — system prompt, user message, model output, reasoning content — is written to an encrypted Vault with its own retention. Usage records land in Postgres, queryable via
/v1/usage. - Budget and billing. Per-service, per-tenant budgets with threshold actions. Credit ledger with reservation and settlement. Cost dashboard that breaks spend down by feature, not just by model.
When you need one
You probably don’t need one for a single internal tool with one LLM call site. Direct calls are fine.
You probably do need one when any of these is true:
- Multiple services call LLMs. Each one reinventing auth and retry is a tax.
- Multiple tenants share your platform. Per-tenant usage and budgets aren’t optional.
- Compliance matters. SOC 2, ISO 27001, GDPR, UK-GDPR — auditors ask where raw prompts live and who accessed them.
- You ship user-facing features. Reply suggestions, chat assistants, code review — these touch customer data, and customer data has PII.
- You want to switch models without a rewrite. A new model ships every week. The ability to reroute without a deploy is a competitive advantage.
How SchneeAI approaches it
SchneeAI is an AI Gateway plus the surrounding platform — prompt registry, cost calculator, token counter, code review integration. Every chat completion goes through the same eight hops: TLS terminate, JWT verify, routing policy, PII scan, LiteLLM provider call, Vault write, usage + audit records, response. See Inside the Gateway for the full walk-through.
The gateway is OpenAI-compatible. If your code calls api.openai.com/v1/chat/completions today, pointing it at api.schneeai.com/v1/chat/completions is a one-line change. The API Request Builder generates the curl / Python / TypeScript / Go snippet for your stack.
Next steps
- Read the architecture: Inside the Gateway — the eight hops in detail.
- Try the request builder: API Request Builder — copy-pasteable snippets in four languages.
- Compare providers: LLM Pricing Benchmark — monthly-refreshed price table across frontier, value, reasoning, and open-source tiers.
- Model costs for your workload: Cost Calculator — model a specific prompt and token count.