From ML Pipelines to Private AI Infrastructure
TensorFoundry has traded since 2018, and its founder brings 25+ years of building software. Each platform below taught us something that shaped what we build and advise on today.
Sherpa
The founder's ML model management, testing, validation and retraining platform, started before TensorFoundry began trading and before "MLOps" was a common term.
Scout (internal platform)
Internal on-premises LLM platform from the early GenAI era. Its routing and model-unification lessons carried into Olla and FoundryOS.
Olla
Open-source LLM proxy and load balancer. One interface for Ollama, LM Studio, vLLM, SGLang, llama.cpp and others, built on what we learnt from Sherpa and Scout.
Kaizen
Terminal-first AI coding agent with persistent memory and a multi-agent crew. Helm coordinates the Scout, Cody and Sage agents, and every change can be rolled back with reversible checkpoints.
FoundryOS
Control plane and LLM router for the self-hosted inference you already run. Load-aware routing across vLLM, SGLang and llama.cpp, with GPU telemetry, health checks and failover. It routes to and monitors your inference endpoints; it does not deploy or schedule them. Early access from Nov 2026; general availability Jan 2027.
What We've Learned
Real-world experience matters. Years of search, machine learning and inference infrastructure work shape the tools we build and the advice we give.
Performance is measured, not claimed. Olla adds about 0.30 ms at the median, and our published benchmarks state the conditions behind every number.
Unification beats fragmentation. One interface across your AI backends means each application integrates once, not once per engine.
Privacy drives adoption. Many organisations need their AI on infrastructure they control, not in someone else's cloud.