Agents and Skills: Finding What Works for Us
Along the way, we've built our own collection of agents and skills, changed how we approach parallel development and got more careful about what we delegate.
We're a small distributed team at TensorFoundry, working across different timezones with different technical backgrounds and interests. We adopted tools like Claude Code early because they've been useful for researching ideas, investigating issues and building our products.
We still spend a great deal of time debating (Marten loves a great debate!) and architecting, but use agents to give us insights, help us prototype, learn and also orchestrate the running of tests (via Skills).
Like most, we started with straightforward tasks. Ask an agent to investigate something, implement a feature, write tests or review a change. For smaller pieces of work it generally went well. You could get something useful done quickly and the results were often pretty good.
Then interesting things started showing up when we gave agents more involved work.
Some of the things we noticed (and I'm sure you have too):
- Code duplication and bloat, common with the 3.x and 4.x Anthropic models, less so with 5.x
- Tests that were flaky, brittle or low-value, they could do with reading Roy Osherove's The Art of Unit Testing
- More time resolving merge conflicts across worktrees because of parallel work, even when it was planned
- Drifting from the task as they find other issues that need addressing
These aren't exactly new software engineering problems, and there are techniques we can employ like Spec-Driven Development.
The difference is that agents can produce a lot of code before you notice you've gone in the wrong direction. We review agent-generated PRs with extra scrutiny and try to keep them small enough to review properly (and also employ tools like CodeRabbit for PRs in Olla).
Seeing these patterns early made it easier to watch out for them and, eventually, to reduce them. We also started looking at how to make the things we already do as engineers more repeatable and automated. That's where our in-house agents and skills library came from.
Agents vs Skills#
The two get mixed up a lot, so here's how we separate them and when we reach for one or the other.
A quick rule of thumb: an agent is good at a particular job. A skill describes a piece of work we want to carry out and how to go about it.
Take our bug-investigator agent. Its job is to reproduce an issue, examine the evidence and test possible explanations. It doesn't touch the fix. The /bug-hunt skill gives that investigation a repeatable workflow: what to brief the investigator with, what to check in its diagnosis and who gets the fix afterwards.
When to use which#
If the work needs the conversation we're having, or a back-and-forth with us, it's a skill. /spec is a good example: the whole point is to capture what we just discussed, and an agent starting from a blank context can't see any of that.
If the work needs fresh eyes, it's an agent. Reviewers, auditors and verifiers all fall into this group - a reviewer that watched you write the code tends to agree with it. The same goes for anything we want to run in parallel, or anything noisy enough that we'd rather it didn't fill up the main session.
More often than not, it's both. /bug-hunt keeps the conversation and the context of the report, while bug-investigator does the actual diagnosis in isolation, so it doesn't just confirm whatever theory came attached to the bug report. And sometimes it's neither - a line in the repo's CLAUDE.md does the job.
Always be looking for the next one#
This is the part I think matters most. Get into the habit of looking at how you work and asking which bits keep repeating. The agents and skills fall out of that.
A few questions we keep asking ourselves:
- Have I explained this to an agent more than twice? That's a skill waiting to be written, or at least a line in
CLAUDE.md. - Is there a checklist I run through in my head? Reviewing a change, cutting a release, checking a spec against the code. If it only lives in your head, it isn't repeatable.
- Is there a test nobody enjoys running? Long-running, fiddly, needs three things started in the right order. These turned out to be some of our best skills, more on that below.
- Do I need a second opinion I can't give myself? Reviewing your own code or your own spec. That's an agent with fresh eyes.
- Is it boring but has to be right? PR descriptions, release notes, SBOMs. That's where
/prand/release-cutcame from.
Not everything should be automated though. What we build, the spec we agree on and what gets merged stay with us. We're after the repeatable parts, so we have more time for the rest.
None of ours started as a grand design. Most began as a fix for something that annoyed us and got refined every time it broke. We keep working through that loop, adding skills and agents as we need them, and the collection has grown organically over the year.
We also keep a close eye on token consumption and try to write efficient skills. That takes a bit of time and practice, but Anthropic's skill authoring best practices has great guidance on how to go about it.
A bit like Agile, really#
Remember the 2010s when everyone was adopting Agile? Except every organisation seemed to do it slightly differently - as a consultant, I adopted so many customer methodologies over time, it was quite fascinating how much variation you'd have. You'd hear "we do Agile our way" quite a lot. Sometimes that meant a team had thought carefully about what worked for them. Other times it meant they'd kept the ceremonies and forgotten why they existed.
I think agents and skills are heading in a similar direction.
There are plenty of collections of prompts, agents and skills available now. Some are very good. You can look at what people like Matt Pocock or the SuperPowers Skills are doing, borrow an idea and have something working in an afternoon. We've certainly taken inspiration from other people's approaches.
But their way of building software isn't necessarily our way of building software - or yours.
A team producing web applications has different concerns from one building Go services, Rust tools, LLM gateways and infrastructure software. Even two teams using the same languages may have very different expectations or views around testing, dependencies, releases and what constitutes a finished change.
Sometimes one of us is researching an idea, someone else is reviewing a change and another is looking at the UX of something we're building. Having shared conventions helps, just as it does when you're working with other engineers. But the conventions need to come from how we actually work, not from a collection of prompts we installed last week.
For us, that's ended up as a few layers, each knowing something the others shouldn't need to.
How ours fit together#
Our shared repository has grown into a collection of specialist agents and workflow skills. At the time of writing it's 20+ agents and 10+ skills. The numbers have changed as we've experimented and I expect they'll keep changing.
We have specialists for Go, Rust, architecture, code review and bug investigation. They know the kinds of things to look for, but they still read the project's own conventions before doing anything. A Go specialist shouldn't assume two repositories use the same patterns just because they're both written in Go.
Skills sit around those specialists. /feature-research helps us explore an approach, /spec turns a settled discussion into something implementable and /orchestrate coordinates the work. We also have skills for /cross-review and other recurring tasks.
Laid out against the way a change moves through development, it looks something like this.
Then there are the skills that belong to a particular repository. That's an important distinction and one we arrived at after trying to make some of them generic.
From an idea to a working branch#
One workflow we use quite often starts with a conversation. We explore an idea, look at the existing code and work through the trade-offs. Sometimes we use /feature-research; sometimes it's just a discussion with a few specialist agents looking into different parts of the problem.
When we've settled on an approach, we run /spec.
This was partly a response to the amount of duplicated code we were seeing. It's easy for an agent to write a perfectly reasonable new component without checking whether something similar already exists. Our spec workflow now looks for adjacent functionality and existing tests before asking an analyst agent to write the specification.
It also tries to preserve what we actually decided. If we discussed an idea but parked it for later, that shouldn't turn into a requirement. If something is still undecided, it should remain a question rather than an assumption the next agent makes for us.
Once the spec is written, we have it reviewed independently. An adversarial reviewer looks at the spec against the codebase without the conversation that led to it. We also use Codex as a separate opinion. The idea is to find the holes before we spend time implementing them.
We then take the agreed spec and hand it to a fresh Claude Code session running /orchestrate.
A recent example is our Flux CLI work. The actual handoff started like this:
/orchestrate docs/specs/flux-v41.md
Context for this run:
- The spec is on branch spec/flux-v41 (head ebb7db58d, on main 6bc7670ed; pushed to origin). Cut feature/flux-v41 from it as the integration branch.
- Read docs/specs/flux-v41-flows.md (the flow document) alongside the spec; the spec's "Where this spec corrects the flow document" list holds only two standing notes. Background research is in docs/research/flux-v41/; the spec wins where they differ.
- Reviews are a fresh Fable code-reviewer plus Codex (gpt-6.1-sol, xhigh) per package and for the final branch; not Fable. Implementers are Sonnet; at most 1 Opus and 2 Sonnet at once.
- One heavy suite at a time on the box (make ready, make test-postgres, any /test-*), at the milestone gates the spec names. Owner rules: robust tests (no timing, core, locale or order dependence), never raise a memfloor ceiling as the fix.
- The gate host needs bash, zsh and fish installed for the completion criterion (AC-UO-27).
- The lab instance needs allow_insecure_http on its flux context; do not deploy to it unless the spec's gates call for it.
- Anything found broken outside the spec goes to /bug-hunt and the final report, not into a package. The rest of the handoff identified the branch to start from, the supporting flow document, the models to use for implementation and review, the test gates and the things the agent wasn't allowed to do. We also made it clear that unrelated defects should go through /bug-hunt rather than becoming surprise additions to the feature.
One line was particularly important: stop when the branch is ready and verified. Don't push or merge.
We're comfortable delegating quite a lot of implementation work, but that doesn't mean the agent gets to decide when something ships.
The orchestrator breaks the spec into smaller work packages, gives those to specialist agents and coordinates review and testing. It also keeps a record of what's been done so the next stage isn't relying on a long conversation that may no longer fit into context.
When that work is finished, we often bring it back to another session, sometimes the one that helped write the original spec, for another review. That review can send us back for corrections or even another small spec. It's not always a straight line from idea to finished feature.
The Flux implementation was still underway when I wrote this, so I'm not presenting it as a completed success story. It's a useful example of how we now hand off a larger piece of work.
Doing less in parallel#
One of the things that sounded great initially was having several agents implement different parts of a feature at once.
For a simple codebase, or pieces of work that really are independent, that can be fine. But as our projects became more complex, it became harder to manage. Agents would touch the same files, work against assumptions another agent had just changed or get tangled up in Git state. We'd spend time resolving conflicts and explaining the new state of the world to everyone. Git Worktrees and dependency management in the spec itself helped alleviate a lot of the friction but we still opt to work serially for a feature on a local implementation.
We've since moved towards smaller, composable implementation packages that run mostly in sequence. That makes it easier to understand what changed, review it and move on to the next piece. It also gives the orchestrator a much clearer job.
We haven't stopped doing things in parallel. Research is a good example of where it helps. Independent reviews are another. Our /design-explore skill produces several different working UI prototypes in parallel, using the same mock data and functional requirements, so we can compare actual design directions. Sometimes the API and UI are separate enough to implement concurrently too.
It's the shared implementation work where we've pulled back.
Why our testing skills became project-specific#
We write quite a bit of Go and Rust, so memory use, goroutines, concurrency and long-running behaviour are things we care about.
We initially tried a generic /test-leak skill. The problem was that it couldn't really know what a leak looked like for every application, or which workloads mattered. An LLM gateway has very different failure modes from a CLI tool. It was also inefficient with tokens as it spends a great deal of time trying to understand the codebase.
We now tend to keep these more involved test skills in the repositories themselves. The shared agents still know how to investigate failures, but the project knows how to exercise its own behaviour.
Alloy, our LLM gateway, is a good example. It has 20+ skills of its own, most of them tests, and they share a small harness for starting the gateway, picking ports and cleaning up afterwards. Its /test-leak skill runs a fairly long set of workloads and examines what happens to memory and goroutines after the work stops. That's an important detail: a goroutine count going up while a server is busy doesn't necessarily mean there's a leak. We want to know what remains once things have settled.
The tests exercise product-specific paths such as streaming clients disconnecting, failover and changes in provider state. There's even a negative control that introduces a known leak to check that the detector actually notices it. A green result is less reassuring if you've never checked that the test can go red.
If you want to see what these look like, Olla is open source and keeps a few of its own. /olla-validate boots a fleet of mock backends and fans out agents to check routing, both API surfaces and failure handling before a release. /test-sticky-sessions checks session affinity across the provider routes and /claude-code-e2e proves Claude Code works turn by turn through Olla against a real Ollama backend.
These skills aren't replacing normal tests. They use scripts, assertions and measurements to establish what happened. Where the agent helps is in interpreting a failure and deciding what to investigate next.
If /test-leak finds something suspicious, we can hand that to the more general /bug-hunt workflow. The bug-investigator agent is told not to fix the problem. It reproduces the issue, tests different explanations, looks at the history and checks whether the same mistake exists elsewhere. Only then does another specialist work on the fix.
We added that separation because jumping from a symptom to the first plausible fix is a good way to miss the underlying problem.
Reviewing with a second model#
Another change that's made a difference for us is reviewing work with a different model.
We use Anthropic models such as Opus and Fable for much of our exploration and orchestration. They're capable, but we've had occasions where a review missed something important, particularly around concurrency, locking or more complicated data structures.
An independent Codex review using Sol has caught issues that the earlier reviews didn't. Sometimes it's a missing failure path, sometimes a concurrency concern and sometimes it's a test we hadn't thought to write.
I don't think one model is better than the other here. They just don't make the same assumptions or look at the same parts of a problem.
There's a catch, though. A model can confidently report a bug that isn't there. We learnt fairly quickly that passing findings between models without checking them just creates more work.
Our /cross-review skill makes a point of verifying the findings against the code. If a reviewer says something can race, we want to understand the actual path that makes it possible. If the code already prevents it, we should be able to show that too.
Yes, that uses more tokens. I'd rather spend them checking something useful than spend a morning chasing a convincing but imaginary defect.
Leaving agents running unattended#
A lot of the less glamorous improvements have come from trying to leave agents working unattended.
A good spec helps because there are fewer open decisions for the agent to guess at. But we've also had to think about what happens when a test runs too long, a command leaves a process behind or an agent asks for permission halfway through an overnight run.
We've had runs stall for hours on questions that could have been answered before we started. That's frustrating when you come back expecting a branch to review.
Our skills now try to make the boundaries clearer: what can be decided without us, what needs to be recorded, which processes must be time-bounded and where the work should stop. We also try not to run heavy test suites over and over after every tiny change.
In our projects, make ready is generally a quick way to check whether the code is in a reasonable state. We run focused tests as work progresses, then the more expensive integration and project-specific suites at meaningful milestones. Running a large suite after every work package can burn a lot of time without telling you much more.
The important part is that an interrupted test doesn't become a pass, and that a result from an earlier commit isn't treated as proof that a later change is fine.
Something we're still working on
Where we've landed, for now#
All five of us use the shared agents and skills, though not always in the same way. One person might use them to explore a design, another to turn a discussion into a spec and someone else to gate a change before it gets reviewed. That flexibility is part of why we've kept working on them.
One useful change has been having a common starting point for recurring work without expecting everyone to work identically. A bug investigation can follow the same broad process across repositories, while the project-specific tests and engineering decisions stay with the people and codebase involved. That doesn't eliminate the need to explain context, but it gives us something concrete to refine together.
We've also changed the workflows repeatedly. Some things that sounded good in July didn't work particularly well by October. The move away from lots of parallel implementers is one example. Moving complex test skills into individual repositories is another. I expect we'll change more of it as the tools improve and we find new problems.
Which brings me back to the Agile comparison.
I don't think there's much value in copying somebody else's set of 20-odd agents and a dozen skills just because they've put a lot of work into them. Yes, I'm aware that's roughly what we have. That's sort of the point - ours fit the way we work, which is the only reason they're useful to us. Read other people's, learn from them and borrow the bits that make sense. But start with the problems you and your team actually have.
Maybe that's making sure a bug is properly diagnosed before somebody fixes it, or getting consistent reviews across a few repositories. Or it's one painful integration test that nobody enjoys running manually.
Turn that into a skill, try it on real work and see what breaks. Change it when you need to. It's the same loop from earlier.
That's been the useful part for us: gradually getting the agents to work the way we already want to engineer software.
If you're interested in the kind of software we're applying this to, have a look at Olla and Alloy.
Where to find inspiration#
A few collections to read before you write your own, for your organisation, your project or just yourself: