Your AI Agents Need an Org Chart

The most productive AI operators I talk to have stopped asking "which model is best?" They're asking a different question: how do I run a fleet of these things without becoming the bottleneck myself? This week's roundtable turned into a working session on exactly that — and the answers sounded less like software engineering and more like management.

The takeaway: Operators getting the most out of AI agents run them like an organization, not a toolbox — departments that send each other memos, a head of HR that owns the policies, a QA function that tests one layer at a time, and a pricing gap between vendors worth measuring while it lasts.

What follows are notes from this week's Executive AI Roundtable discussion, shared under the Chatham House Rule.

Send Memos, Don't Grant Logins

One operator runs a couple of dozen concurrent AI initiatives — each one a repository with its own agents working overnight shifts. The coordination problem looked familiar to anyone who has managed people: how do projects share progress and request changes from each other without stepping on each other's work?

Their answer is deliberately bureaucratic: agents don't edit each other's repositories — they file a GitHub issue on the other repository instead. They call it an inter-office memo. The overnight run on every project starts by draining its inbox of issues, acting on them, and filing memos back when the work is done. "Don't go directly editing the other guy," as they put it. "Send them a memo by putting up the issue."

The alternative — every agent with write access to everything — produces a failure they described as "middle managers writing policy for the company": one project's agent, solving one project's problem, quietly rewriting a shared standard that every other project depends on. The memo system buys indirection: requests become visible, reviewable, and orderable. Peer-to-peer coordination happens without a human couriering messages between terminals.

The same operator has been rereading Andy Grove's High Output Management — forty-year-old Intel doctrine — and applying it to agents: "I thought it'd be very useful for thinking about AI." Separation of concerns, spans of control, and indirection. The org chart is back, and this time the reports are tireless. Running a mixed roster of people and agents is its own discipline — more in managing a workforce of humans and AI agents.

Improve the Skill, Not the Memory

When something goes wrong in one project — an agent misses a check, a review lets a bug through — the instinct is to fix it there and move on. One operator refuses: they keep a separate repository that holds every skill their agents use, and route all process fixes through it. They call it their operating model — "my head of HR."

The rule: "Let's improve the skill, not our memory." A fix recorded in one project's memory helps that project. The same fix pushed into a shared skill helps every project that loads it, session after session: "Each thing I'm doing is giving me compounding value across my projects, and each time I do a clear, it's getting access to the skills yet again, and sort of keeps on cycling up."

When one project's agent wants a harness-level policy changed, it files a memo with the operating model, which weighs the request against feedback from every other initiative before making any changes. Individual projects don't get to rewrite company policy. That's the head of HR's job.

More on skills as the unit of compounding in AI skills that replace workflows.

The discussion also drew an autonomy line. Micromanaging agents with constant check-ins doesn't scale; neither does unsupervised chaos. The calibration on offer was a driving metaphor: "Go ahead. You go do the wrong thing. Go drive yourself into the ditch. And call me when you're in the ditch." Pulling an agent out of a ditch occasionally costs less than checking on it every mile.

Give the Org a Rival Auditor

My own contribution to the org chart is the auditor's office. I open-sourced a Claude Code skill called Impasse: Claude does the work, then hands it to OpenAI's Codex — a rival model from a rival company — for a hostile review. The two argue the findings out between themselves, and only the disagreements neither can settle with evidence reach me.

Claude will present a change that it is fully satisfied with — and Codex is the one that says, in effect, "hey, no, wait a minute." One attendee put the underlying idea this way: "You got these two big giants are making these great models... they will make each other better if you just allow them to."

I've pointed it at code, at writing, at website changes — and this week at a client proposal, where it caught errors the drafting model had already blessed. The reasoning behind cross-provider review is in AI's second opinion.

An App-Building Machine on a Spare Login

One founder demonstrated what a fully-staffed agent org looks like on a small scale. A family member wanted changes to an app the founder had built for them. Instead of fielding requests, the founder wired Claude to Apple's TestFlight feedback channel and gave it a standing instruction: every thirty minutes, check for new tester feedback, action it, and ship a new build.

"For a couple of days," they said, the family member "was giving feedback, getting new builds... without my involvement, a hundred percent." The whole thing ran in a terminal on a separate macOS user account they occasionally checked on — "like an app-building machine."

The setup generalizes past apps: a feedback source agents can read, a standing cadence, a delivery channel, and a human who owns the outcome but not the loop.

Put a Cockpit Between You and the Agent Fleet

Span of control is an interface problem. On my workstation, I've abolished alt-tab: a Stream Deck gives every tmux session its own button, with an indicator that changes color and spins while Claude is busy — a glance tells me which agents need me and which don't. Dials move focus between panes; a foot pedal toggles dictation so instructions flow while my hands stay on the work. Claude built every one of those plugins, and I've open-sourced them.

The input side matters as much as the display. One operator has dictated over 600,000 words to their agents since May because, as they put it, "you can unload a lot more of your brain in a lot less time." Typing was the bottleneck between manager and organization; they removed it.

Test the Layer, Not the World

One founder brought a testing problem that AI-built products keep hitting. Their multi-stage pipeline — video goes in at one end, structured text comes out the other, agents act on the result — was giving them mostly end-to-end confidence: plenty of tests and logs, but when something broke, the signal arrived at the far end without saying which stage had failed. Worse, every trivial change triggered the full gate suite — a thousand tests, a forty-package type-check, and a production build — before a human could even look at the result.

Two experiments the room converged on:

Per-layer evals with fixed fixtures. Keep one fixed input that should always produce roughly the same output, and test that layer alone against it — exact wording will vary, because it's AI, but the key items shouldn't. Give each downstream stage the same treatment: a known starting state, an expected result, and an assertion that the stage still delivers the expected result. When something regresses, you know which stage did, instead of re-litigating the whole pipeline.

A feature-testing skill that owns the test scope. Write the judgment down once, as a skill the project invokes on every change: write tests for what changed, run only the directly-impacted tests, and reserve the full suite for merge day or a scheduled overnight run that files its findings as morning to-do items. The human's hands-on pass stays the last gate — on my projects, the final sign-off is me testing the feature end-to-end.

The founder's instinct — "production is essentially my dev site" — is honest for a pre-launch product, and the right response is to write that stage into the project's context so the agents stop enforcing ceremony the stage doesn't need. Let agents commit and push constantly — my own executive-assistant system commits after every unit of work. A huge commit history stays cheap to search, because git bisect can binary-search a thousand commits in about ten test runs — provided you keep a reliable reproducer and buildable commits.

The founder said it plainly: "My prototype was working way better than the SaaS that I'm building now." The agents are fast; the surrounding process — testing, review, deployment — is where the week actually went. That's the argument for giving them an organization.

Keep the Org Out of iCloud

Agents generate and delete enormous numbers of small files — work trees, build artifacts, node modules. Pointed at a folder inside iCloud sync, that workload cost one founder days: the sync engine jammed outright, half a million items queued, "stuck for days," the machine consumed by a backlog it could never finish. Sync services are built for humans editing documents, not for agents churning a hundred gigabytes of scratch space. Keep agent workspaces in plain Git repositories outside any synced folder — the repo is also your recovery story when an agent goes off the rails.

The Pricing Gap

The session closed on economics. One founder who routes work across both vendors measured an overnight run: the coding, done by Codex, consumed about 3% of its weekly quota; the reviewing, done by Claude, consumed about 10% of its own. The meters aren't measuring the same work, and nobody on the call could say exactly why — orchestration overhead was one guess. We speculated that OpenAI is secretly under-metering Codex in their bundled accounts to win the agentic-coding market. That part is a guess. What isn't a guess is how much they want Codex adopted: the plugin they shipped for Claude Code is a distribution gambit built inside a rival's product.

For an operator, the move is an experiment with eyes open. Route heavy coding work to whichever meter your own measurements say is cheaper — the room's version was to use the heck out of Codex while it lasts — and architect so the router, not the model, is the part you own. Pricing windows close. Org charts survive re-orgs. Why the harness outlasts any one model: the AI harness beats the model.

Where to Start

• Pick one process, identify an improvement you'd like, and push it into a shared skill instead of a chat correction — that's the difference between a lesson and a policy.
• Split one always-run test suite into per-layer checks with a fixed fixture each, and move the full suite to a nightly run that files its findings.
• Set an autonomy line for your agents — decide what "call me when you're in the ditch" means for your work, and stop checking every mile.
• Audit where your agent workspaces live — move anything inside a synced folder into a plain Git repository today.
• Add a second, rival model as a reviewer on your next consequential piece of work, and route only the disagreements to yourself.

Resources From the Roundtable

• Impasse — my open-sourced Claude Code skill for cross-provider adversarial review, discussed as the session's auditor pattern.
• High Output Management — Andy Grove's management classic, cited as the framework for agent orchestration.
• TestFlight — Apple's beta distribution channel, the feedback loop behind the app-building machine.
• tmux — terminal multiplexer; detach-and-reattach keeps sessions alive while the machine stays awake (pair it with a keep-awake tool).
• Amphetamine — macOS keep-awake utility for long unattended agent runs.
• Elgato Stream Deck — the hardware behind the agent cockpit: one button per session, live busy indicators, a foot pedal for dictation.
• git bisect — binary search over commit history; why a huge agent-generated commit log stays cheap to debug.