Olla v0.1.0: TEI, New Dashboard & Fixes

Olla v0.1.0 marks a milestone for the project. The repo was created in May 2025, about 18 months ago, and this is our 25th GitHub release. It is also our way of acknowledging everyone who helped get Olla here: the people who sent pull requests, filed issues and ran it on their own hardware. Thank you to every contributor.

The release adds a new backend type, reworks the dashboard, hardens rate limiting and concurrency, and corrects request stats. Several of those change what existing setups see, so read Before you upgrade first. Fixes from here ship as v0.1.x.

Text Embeddings Inference backend

Olla now supports Hugging Face Text Embeddings Inference (TEI) as a backend, with embeddings, rerank and classification routes, model kinds taken from TEI's /info, and /olla/tei/ provider prefixes. It was contributed by @MartinEmrich (#231), thank you.

Rerankers and embedders are recognised by name on backends that report no kind (#248), and /olla/models now exposes the kind as olla.type (#249)
A request that names no model is narrowed by capability. If the only capable backend is down it gets a 503 rather than landing on the wrong backend (#236)
Ollama /api/embed and embedding models route correctly in a fleet that includes TEI (#247)

A reworked dashboard

The Endpoints panel gets selectable herd columns, a column picker, an issues tooltip and a filter (#265). The wordmark and header layout were also fixed so the Overview holds up on narrow screens (#265).

Olla dashboard Overview showing fleet status, a requests-per-second sparkline and the herd table of backends — Overview: fleet status, live throughput and the herd at a glance.

Olla dashboard Endpoints panel listing each backend with its health, priority, latency and model count — Endpoints: per-backend health and detail, with the new columns and filter.

Reliability and stats

New proxy.retry block controls failover: enabled, max_attempts and non_idempotent. The defaults match the old hardcoded behaviour, so nothing changes until you set it (#260)
Request stats are corrected: each request counts once per backend attempt, and a backend 5xx is a failure (#267)
Model discovery retries failed fetches (#245), tolerates bad responses, and no longer sends backend credentials to HTTP_PROXY or across hosts (#223, #234)
Fixed a crash under concurrent model discovery and requests (#233), and a model hosted on several backends no longer flips its listed id (#246)
Unknown keys in config.yaml are reported at startup, once each (#258, #263), and the config and docs no longer promise settings Olla ignores (#255)
Services stop in reverse order when startup fails, thanks to @SDTBU-FXL (#225)

Rate limiting and request handling

The global limit and the per-IP limit are independent tiers, checked per-IP first, and every 429 reports zero remaining. @matthewjhunter fixed the global limiter being skipped by the per-IP short circuit (#256)
One client can no longer flood the log with rejections (#237)
Chunked bodies on Anthropic routes are capped at the translator limit (#261)
Model routing no longer depends on the Content-Type header, and Content-Encoding: identity bodies are read (#238)
/olla/models/{id} works for IDs containing slashes (#244), and Ollama model names without a tag resolve to :latest (#235)
Ollama vision and tool capabilities are read from the backend (#250), and LM Studio model types and capabilities are reported correctly

Before you upgrade

These are the changes that can alter what an existing setup sees. None need a code change unless the right-hand column applies to you.

Olla v0.1.0 behaviour changes and what to check
What changed	What to check
Request counts can halve, and a backend 5xx now counts as a failure (#267)	v0.0.29 counted every request that named a model twice, and earlier releases counted a 5xx as a success. total_requests, success_rate, total_failures and the matching olla_* metrics now count each request once per backend attempt, so status verdicts and any alert on them may change.
Anthropic and Claude Code routes are now rate limited (#257)	/olla/anthropic/v1/messages, count_tokens and /models share the proxy limits and each client's per-IP bucket. Many sessions behind one IP (NAT, CI, parallel agents) may see 429s, so raise per_ip_requests_per_minute and burst_size.
per_ip_requests_per_minute: 0 no longer disables rate limiting (#256)	The global limit (default 1000 per minute, burst 50) still applies. Set global_requests_per_minute: 0 as well to turn both off.
No retry once part of the response has reached the client (#260)	A GET that failed mid-stream used to be replayed on the next backend. It now ends with the error.
SGLang and LMDeploy pass Anthropic requests through (#242)	Needs SGLang v0.5.9+ or LMDeploy v0.13.0+. For older builds, override the profile (zz-sglang.yaml or zz-lmdeploy.yaml) with api.anthropic_support.enabled: false.
Profiles declare the paths they serve (#240)	Shipped profiles now list /v1/responses, /v1/messages, /v1/rerank, /api/ps and /api/version. A custom profile serving these paths needs the same entries.
Anthropic translation uses each backend's own chat path (#241)	Docker Model Runner gets /engines/v1/chat/completions and Lemonade /api/v1/chat/completions. If no backend serves a chat path, the request gets a 404.
Ollama vision and tool models are typed differently (#250)	Vision models are vlm with the vision capability, and tools maps to function_calling. ?type=llm no longer lists vision models; ?capability=chat still does.
/olla/ollama/api/ps merges every Ollama host (#243)	Entries carry olla_endpoint, a partial answer sets X-Olla-Partial and the route is rate limited. The /olla/ollama/api/list stub is gone and /api/blobs returns 501.
Log lines and headers (#253, #262)	"Request completed" lines carry http_status and a backend 5xx logs at WARN instead of INFO, so check log-based alerting. X-RateLimit-* headers are omitted when no rate limit applies.
Config and models.yaml (#245, #248)	Discovery retry_attempts and retry_backoff are now honoured, and a negative retry_backoff fails startup. A custom models.yaml with its own name_patterns needs the two new entries for rerankers and embedders.

One addition is safe to ignore: olla.type in /olla/models and a top-level type on /olla/models/{id} (#249). Run olla --validate-config against your config before you roll it out.

Thanks

@MartinEmrich added the Text Embeddings Inference backend (#231)
@matthewjhunter fixed the global rate limiter and reported the retry-on-partial-response, chunked-body cap and unknown-config-key issues
@SDTBU-FXL fixed the service rollback order on failed startup (#225)

Upgrading to Olla v0.1.0

Run --validate-config before every upgrade. It checks config.yaml, models.yaml and every provider profile without starting the server, and exits non-zero if anything fails. We want everyone to do this first.

Pull the new image

$ docker pull ghcr.io/thushan/olla:v0.1.0

Validate your existing config with the new version

$ docker run --rm \
  -v "$(pwd)/config/config.local.yaml:/config/config.yaml:ro" \
  -e OLLA_CONFIG_FILE=/config/config.yaml \
  ghcr.io/thushan/olla:v0.1.0 \
  --validate-config

Result: PASS - all configuration files are valid

Only then, stop the running container
```
$ docker rm -f olla
```

Start it on the new image

$ docker run -d --name olla -p 40114:40114 \
  -v "$(pwd)/config/config.local.yaml:/config/config.yaml:ro" \
  -e OLLA_CONFIG_FILE=/config/config.yaml \
  ghcr.io/thushan/olla:v0.1.0

The image runs olla as its entrypoint, so extra arguments go straight to it. Adjust the mount path to wherever your config lives, or grab a binary from the release assets and run olla --validate-config directly. The full list of changes, with a link for each, is in the v0.1.0 release notes.

Resources

GitHub Release Full release notes on GitHub

Release Notes Upgrading checklist and every change

Olla Documentation Setup, backends and configuration

Olla Product Page Learn more about Olla