One gateway. Every provider.

Alloy is an enterprise LLM gateway and control plane. It sits between your applications and every major cloud and local LLM provider, and gives you central API key management, virtual models with failover, team and key budgets, cost tracking, prompt guardrails and audit logging. It ships as a self-hosted single binary with an admin Console, the Prism self-service portal and the Flux CLI.

Engineering teams and organisations that use more than one LLM provider and need control over who can call what, and what it costs. It suits teams whose provider keys are scattered across CI pipelines and laptops, or where finance needs spend attributed by team and key.

Alloy 4.0 LTS is available to early-access customers now, with public release at the end of November 2026. Request access and we will get you set up and take your feedback into the release.

Alloy supports dozens of provider types, covering the major cloud providers, including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Vertex, Gemini, DashScope, Mistral, Groq, DeepSeek, xAI, Together and OpenRouter, and local or self-hosted servers, including Ollama, vLLM, llama.cpp, SGLang, LM Studio, Lemonade and Docker Model Runner. Custom OpenAI-compatible and Anthropic-compatible endpoints work too. Clients talk to Alloy over OpenAI chat completions, Responses, embeddings and moderations, or the native Anthropic messages API, so tools such as Claude Code work out of the box.

A virtual model is one stable name, such as qwen3.8-27b or code-assist, that your applications put in the model field. Behind it sit one or more deployments, each a provider with its own model id, priority and weight. Alloy picks a deployment by priority, round-robin or weighted share, so you can fail over, split traffic, or swap vendors without changing any client. Model access can be limited per team and per key.

If a deployment fails in a transient way, such as an upstream error, a timeout or a payment error, Alloy moves to the next deployment in the virtual model, within a hop limit you configure. Failover happens before the first byte is streamed, and a client's own bad request is never retried elsewhere. Circuit breakers sideline unhealthy deployments until they recover, and provider key pools rotate keys when one is rate limited.

Set USD budgets per team and per key over a period, plus RPM and TPM limits per key, team and deployment. Observe mode shows what a limit would have done before you enforce it, and email or signed-webhook notifications warn you as spend approaches a limit. Spend is recorded after each request, so the next request is the one that is denied. Every request records its cost and where the price came from, so you can trace a number back to its source.

Yes, on the request side: prompts are screened before they leave your network, and responses are not inspected. Use built-in regex rules, self-hosted Presidio or LlamaGuard, or the Lakera and OpenAI moderation cloud services, and choose to block, redact (regex and Presidio) or log.

Olla is our open-source local proxy: it load balances across local inference backends such as Ollama, vLLM and llama.cpp. Alloy is the governed gateway for both cloud and local providers, adding API keys, budgets, policy, identity and audit on top. Teams often run Olla at the edge and use Alloy where they need control and accountability.

Yes. One alloy binary serves the gateway, Console and Prism with the UI embedded, on SQLite with no external services. The licence is an offline signed token validated locally, pricing rate cards are refreshed by uploading a release file, and local providers such as vLLM, Ollama, llama.cpp and SGLang need no internet.

Under 0.3 ms at the median on a single request stream: 0.17 ms streaming, 0.18 to 0.22 ms non-streaming and 0.27 ms on the Anthropic Messages API. That was measured on Alloy v4.0 against an instant mock upstream on one AMD Ryzen 9 5950X 16-core desktop, with every request going team key to virtual model to provider, on SQLite with logging and cost tracking on. The same desktop sustained over 40,000 requests per second through Alloy, idling at about 70 MB of memory, with zero errors across more than 34 million proxied requests. A real model typically takes 200 ms or more to its first token, so Alloy adds a small fraction of a request.

Request access through the Alloy waitlist and we will be in touch about early access. Request access.