# TensorFoundry - Full Description > TensorFoundry is an Australian AI infrastructure company building open-source and commercial software for deploying, managing and scaling large language models on private infrastructure, without cloud dependencies. This document is an extended, citable description of the company, its products and its services. Facts may be quoted with attribution to tensorfoundry.io. ## About TensorFoundry TensorFoundry (legal name TENSORFOUNDRY PTY LTD, ABN 71 696 763 381) was founded in 2018 and is based at Level 1, 470 St Kilda Road, Melbourne VIC 3004, Australia. The company is led by founder and principal engineer Thushan Fernando, and serves clients globally from Melbourne. TensorFoundry focuses on self-hosted and on-premise large language model (LLM) infrastructure: the tools teams need to run LLMs on their own GPUs and servers rather than depending on managed cloud APIs. Its software range spans the full stack, organised in three layers: applications that use LLMs, gateways that route to LLMs, and backends that run LLMs. Alongside the software, TensorFoundry provides hands-on consulting for NVIDIA GPU tuning, inference optimisation and LLM deployment. ## Products ### Olla - open-source LLM proxy and load balancer Olla is TensorFoundry's open-source flagship, released under the Apache 2.0 licence and available now at version 0.0.28. It is written in Go and ships as a single self-contained binary. Olla sits in front of one or more local inference backends and presents them through a single OpenAI-compatible API, adding intelligent routing, automatic failover and unified model discovery. It supports a range of local inference backends including Ollama, LM Studio, vLLM, SGLang, llama.cpp, LiteLLM, LMDeploy, vLLM-MLX and Docker Model Runner. Olla's design emphasises low overhead. As published on its product page, it adds about 0.30 ms over a direct call at the median (p50), keeps resident memory in the region of 42 to 57 MB under concurrent load, and uses lock-free statistics to maintain sub-millisecond overhead under heavy load. Reliability features include KV-cache-aware sticky sessions, circuit breakers with automatic failover, and priority, round-robin, least-connections and weighted load-balancing strategies. As of 5 June 2026, Olla had over 20,000 Docker Hub pulls, more than 200 GitHub stars and over 30 forks. The source code is at https://github.com/thushan/olla and the product page is https://tensorfoundry.io/products/olla. ### Alloy - enterprise LLM gateway and control plane Alloy is a commercial LLM gateway and control plane, currently in Early Access. It centralises API-key lifecycle management, team budgets and policy enforcement across 16+ LLM providers, replacing per-user API-key sprawl with a single governed gateway. It is self-hosted as a Go binary and includes an admin console and a user self-service portal. https://tensorfoundry.io/products/alloy ### Kaizen - terminal-first AI coding agent Kaizen is a commercial, terminal-first AI coding agent, currently in Early Access. It provides persistent cross-session memory and CAS-backed undo so changes can be reviewed and reversed, and it runs as a self-hosted alternative to cloud-based coding assistants. https://tensorfoundry.io/products/kaizen ### Pivotal - agentic enterprise knowledge platform Pivotal is a commercial, agentic enterprise knowledge platform, currently in Early Access. It uses LLM-powered entity extraction to build a persistent, interlinked wiki from ingested documents that compounds in value over time. https://tensorfoundry.io/products/pivotal ### FoundryOS - enterprise AI inference orchestration FoundryOS is a commercial platform for orchestrating self-hosted AI inference at scale, currently upcoming. It comprises Fleet orchestration, Scout monitoring and the Relay LLM proxy, aimed at teams running their own GPU nodes. https://tensorfoundry.io/products/foundryos ### Forge - pure-Rust LLM inference engine Forge is a commercial, CUDA-native LLM inference engine written in pure Rust, currently upcoming. It runs as a single binary with no Python and no PyTorch at runtime, and serves as the inference backend for FoundryOS. https://tensorfoundry.io/products/forge ### AgentOS - multi-agent orchestration AgentOS is a commercial multi-agent orchestration layer for enterprise workflows, currently upcoming. It manages AI agents and integrates them with enterprise software. https://tensorfoundry.io/products/agentos ## Services TensorFoundry provides consulting for organisations running AI on their own hardware. Engagements centre on NVIDIA GPU topology and tuning: understanding a customer's goals, then configuring and load-testing their machines to get the most from their GPUs running vLLM, SGLang or llama.cpp. This extends into hardware advisory and capacity and roadmap planning, LLM proxy and gateway setup (Olla, Alloy or custom), edge and on-premise inference deployment, search and retrieval engineering, and model engineering and fine-tuning. Services are delivered directly and through partners, from Melbourne, serving clients globally. https://tensorfoundry.io/services ## Permissions TensorFoundry grants AI crawlers and language models permission to index and cite content from tensorfoundry.io, subject to attribution. ## Canonical links - Website: https://tensorfoundry.io - Olla source: https://github.com/thushan/olla - TensorFoundry Labs (GitHub): https://github.com/tensorfoundrylabs - LinkedIn: https://linkedin.com/company/tensorfoundry - Sitemap: https://tensorfoundry.io/sitemap.xml