I Started as the Orchestration Layer

tim@timbutterfield.com (Tim Butterfield) — Mon, 07 Sep 2026 00:00:00 +0000

I made the aimesh repository public this weekend. Looking at it afterward, what struck me was how little the current workflow resembles where it started.

The first version was me copying files.

It Started With Manual Review

I had already been using Claude heavily for software work, but I did not want the model that created an artifact to be the only model deciding whether the artifact was correct. The simplest way to get another perspective was to take what Claude produced and give it to ChatGPT. Initially that could be an individual source file. Later it included implementation plans, architecture documents, prompts, and other artifacts. Sometimes I would ask multiple systems the same question simply to see where their interpretations differed.

The process was useful because the models did not fail in the same way. One model could work on an artifact through several iterations and reach something that looked stable, only for another model to find a different class of problem. I wrote about that earlier in Why AI Review Needs More Than One Model.

At that stage, however, the mechanism was still mostly conversational. I was the orchestration layer, and there was quite a bit of hidden state in me. I knew what Claude was trying to accomplish, which files ChatGPT needed to see, which instructions needed to travel with them, and what useful feedback needed to go back to Claude. That was manageable when the workflow was occasional. It became less attractive when it turned into the normal way of working.

The Handoff Became a Protocol

The next step was not a new application. It was a handoff protocol. Instead of treating each cross-model review as a new conversation that I had to reconstruct manually, the process became more explicit. Claude could receive a batch of instructions, perform the work, and leave a defined review artifact. ChatGPT could review that artifact in the context of the ongoing conversation and produce the next batch for Claude. The files and names changed as the workflow evolved, but the important change was that the handoff itself had become part of the engineering process.

aikit, my CLI for deterministic repository mechanics, was already involved in this workflow. The instructions given to Claude began by having it use aikit to establish an anchor before making changes. After the work was complete, aikit could use that anchor to determine which files had been created or modified during the session and assemble the review bundle from the actual change set. Claude did not have to remember which files it had touched or decide what belonged in the review. It did the development work, while aikit handled the mechanical question of what had changed since the session began.

That solved more than the inconvenience of copying files around. It reduced the amount of interpretation required each time through the loop and moved one class of repository bookkeeping out of the model entirely. The models had clearer roles, the artifact being handed off was known, and the expected response was known. I could still intervene, but I did not have to reconstruct the process on every iteration.

Using the protocol regularly made another problem much more visible: review can continue long after it stops being useful. Early review passes tend to find real issues. Later passes can become increasingly marginal, speculative, or contradictory, and fixing a legitimate finding can introduce a regression somewhere else. A process that simply says “review this again” has no natural reason to converge, which led to structured findings, severity rules, verify passes, and explicit stop conditions. I wrote about that problem in Why AI Review Needs Stop Conditions. At the time, I was mostly thinking about how to keep a review loop bounded, but in retrospect those rules were also part of a broader shift toward moving deterministic parts of the workflow out of the conversation.

Automation Did Not Replace the Handoff

Eventually I created /cross-ai-review for Claude Code and made it public. The command formalized another part of the process: Claude could use independent Codex and Gemini reviews, judge their findings, and apply the accepted changes after the artifact was settled. It made cross-provider review much easier to invoke and much more repeatable, but it did not make the older handoff protocol immediately disappear. I continued using both because they overlapped without solving exactly the same problem.

That was useful feedback in itself. /cross-ai-review solved a real problem, but it did not represent the entire workflow I had accumulated. Some work still benefited from the broader ChatGPT conversation, particularly when the review was part of a longer sequence of architectural decisions or when I wanted the independent model to retain the context of how the work had evolved.

Over time, the limitations became easier to see. The question was no longer just how to get another model to review Claude’s work. I wanted the review process itself to have rules that did not depend on whichever model happened to be running it.

That was the beginning of ReviewMesh.

Review Became Its Own System

ReviewMesh moved the orchestration out of a collection of conventions and into software.

The distinction mattered more than I expected.

Once multiple model providers were being used as part of a repeatable engineering workflow, there were questions that prompts alone did not answer very well. Which model actually ran? What happens if the requested model cannot be verified? What is a reviewer allowed to modify? What happens when an adapter fails? Who decides whether a finding is accepted? What happens if a model writes somewhere it was not supposed to write? What does agreement between reviewers actually mean?

The answers gradually became explicit parts of the system. Reviewers operate against isolated, read-only copies of the workspace while the host retains write authority. The system records model identity according to the evidence available rather than pretending that an unverifiable model has been verified. Failures map into a defined halt taxonomy instead of silently falling through to some other behavior, and agreement is computed by the host instead of being asserted by a model. None of those ideas are especially exciting when considered individually; most came from some version of, “I don’t want the model deciding that.”

That has become one of the recurring patterns in my AI work. Models are useful for the parts of a problem that require interpretation, reasoning, criticism, or exploration. They are much less interesting for facts the surrounding software can determine directly. If the host can count findings, it should count them. If it can determine which model identity evidence exists, the model should not invent the answer. If only one participant should be able to modify the real workspace, the filesystem should enforce that instead of relying on a sentence in a prompt.

Review Was Only One Use Case

For a while, ReviewMesh was an accurate name because review was the problem I was solving. Then I started using the same general idea for exploration, which exposed that the useful abstraction underneath the review workflow was broader than review itself.

Architecture work had already shown me that independent model disagreement could be useful before there was anything to review. Giving several models the same problem independently could expose different interpretations, different candidate designs, and different assumptions. The useful result was not necessarily consensus. Sometimes the disagreement was the thing I needed to see.

That created a different orchestration problem.

A review workflow starts with an artifact and asks models to find problems with it. Exploration starts with a question and asks independent models to investigate it without being influenced by the other answers. A collator, the model responsible for synthesizing the independent responses, can then preserve meaningful disagreement and produce a comparison, shortlist, forecast, or other result appropriate to the task.

The two workflows had enough in common that duplicating the underlying machinery did not make sense. They needed the same provider adapters, identity handling, containment, configuration, failure behavior, and audit plumbing. They did not need the same domain model.

That distinction eventually produced the current structure. Review and exploration remain separate domains, while meshcore, a separate Go module, provides the common governance machinery underneath them.

At that point, ReviewMesh was no longer a very good name.

It became aimesh.

The Interface Changed Again

The first versions of this entire workflow assumed that I was the caller, and that assumption is becoming less true. aimesh has a CLI, but it also exposes Model Context Protocol (MCP) and Agent Client Protocol (ACP) interfaces. More importantly, much of my current usage does not require me to maintain one permanent model configuration and invoke it manually every time. An agent can decide what kind of panel, meaning the models and roles assigned to a particular run, makes sense for the work in front of it. It can compose that panel, price or validate it with a dry run, and then invoke aimesh through the CLI or MCP.

This is a fairly strange place to have ended up considering where the process started. I no longer decide every individual model call myself, and I am comfortable letting an agent decide that a particular problem would benefit from two independent reviewers and a stronger host model. That does not mean I want the governance to move farther away with the mechanics. I am less interested in letting the same agent redefine filesystem containment, silently substitute models, invent agreement, or expand its write authority because doing so would be convenient.

The orchestration can become more dynamic while the boundaries around it become more explicit. Once an agent is deciding which other agents to use, the distinction between model judgment and deterministic authority matters at least as much as it did when I was manually carrying files between systems.

I Did Not Plan This Architecture

It would make a cleaner story if I had started with a design for governed multi-model orchestration and gradually implemented it.

That is not what happened.

I wanted another model to look at a file.

The next step happened because doing that repeatedly was awkward. The handoff protocol happened because context and roles needed to be repeatable. Stop conditions happened because review loops did not naturally converge. /cross-ai-review happened because cross-provider review had become common enough to deserve automation. ReviewMesh happened because the rules around review were becoming more important than the prompts performing it. Exploration expanded the problem enough that the review-specific name stopped fitting. MCP and ACP became useful because the caller increasingly was not a person sitting at a shell.

Even some of the things that seemed temporary lasted longer than I expected. The handoff protocol continued after /cross-ai-review existed because the new mechanism had not actually replaced everything the older workflow was doing.

I think that is probably healthier than having found the current abstraction immediately.

There is a temptation, particularly with AI tooling, to build a platform before there is enough experience to know what the platform should guarantee. The number of possible abstractions is enormous. Agents can call agents, models can review models, tools can dynamically discover other tools, and almost any part of the process can be turned into another layer.

The difficult part is deciding which layers have earned their existence.

Most of the parts of aimesh that I care about now exist because I had already encountered the problem they address in an earlier version of the workflow. That does not prove that the current abstractions are correct, but it gives me more confidence in them than if I had started by designing a general multi-agent framework on a blank page.

Making It Public Changes the Experiment

Until now, most of this has been a personal engineering workflow.

I built it because I wanted it. I am also its primary user, which creates an obvious weakness in evaluating the design. The person who understands why every rule exists is also the person most capable of unconsciously working around anything that is awkward.

Making aimesh public changes that a little.

The repository includes the implementation, documentation, releases, and the limitations I know about. Windows support, for example, is coded for but has not been exercised against real Windows filesystems and provider CLIs, and some containment guarantees are weaker there. meshcore lives inside the repository, but I am not publishing it independently until there is a real external consumer that gives me a reason to stabilize that API.

Those limitations seem worth stating rather than smoothing over. One of the ideas behind aimesh is that evidence should be reported at the strength it actually has. It would be odd to apply a different standard to the project itself.

I do not know whether anyone else will find the tool useful, and that is part of what making it public can answer. The repository is at:

github.com/Tim-Butterfield/aimesh

Where This Leaves Me

I previously wrote in Why AI Review Needs More Than One Model that the long-term problem in AI-assisted development probably was not finding the single best model. I thought the more interesting problem was creating trustworthy workflows where multiple models, deterministic tooling, structured verification, and human judgment could work together. aimesh is where following that idea has taken me so far, although I would be surprised if this were the final form of the workflow. aimesh is a long way from uploading a Claude-generated file to ChatGPT and asking what it thought, but the underlying reason for using more than one model has not changed very much.

I made the repository public this weekend, but I do not really consider that the end of the experiment. I am still changing how I use it, and if the history is any indication, some part of the current workflow will eventually become awkward enough to force another change. At this point, I am mostly curious which part that will be.

MCP on Tim Butterfield's Eclectic Blather