OUR JOURNEY

From ML Pipelines to Private AI Infrastructure

TensorFoundry has traded since 2018, and its founder brings 25+ years of building software. Each platform below taught us something that shaped what we build and advise on today.

2016-2023

Sherpa

The founder's ML model management, testing, validation and retraining platform, started before TensorFoundry began trading and before "MLOps" was a common term.

Traditional ML Model Versioning Pipeline Automation Model Retraining
2023

Scout (internal platform)

Internal on-premises LLM platform from the early GenAI era. Its routing and model-unification lessons carried into Olla and FoundryOS.

vLLM On-Premises AI Model Unification Inference Load Balancing
2025

Olla

Open-source LLM proxy and load balancer. One interface for Ollama, LM Studio, vLLM, SGLang, llama.cpp and others, built on what we learnt from Sherpa and Scout.

Open Source Multi-Backend Community Driven
2026

Kaizen

Terminal-first AI coding agent with persistent memory and a multi-agent crew. Helm coordinates the Scout, Cody and Sage agents, and every change can be rolled back with reversible checkpoints.

Multi-Agent Persistent Memory Early Access
Nov 2026

FoundryOS

Control plane and LLM router for the self-hosted inference you already run. Load-aware routing across vLLM, SGLang and llama.cpp, with GPU telemetry, health checks and failover. It routes to and monitors your inference endpoints; it does not deploy or schedule them. Early access from Nov 2026; general availability Jan 2027.

vLLM SGLang llama.cpp GPU Telemetry

What We've Learned

Real-world experience matters. Years of search, machine learning and inference infrastructure work shape the tools we build and the advice we give.

Performance is measured, not claimed. Olla adds about 0.30 ms at the median, and our published benchmarks state the conditions behind every number.

Unification beats fragmentation. One interface across your AI backends means each application integrates once, not once per engine.

Privacy drives adoption. Many organisations need their AI on infrastructure they control, not in someone else's cloud.