Agent Harnesses are Just Shells

What does an agent harness do? It of course offers the AI model a set of tools it can invoke. Amp has 22 right now, including Bash to run commands, a couple for manipulating files (create_file, edit_file, undo_edit) a couple for doing search (grep, glob, finder, librarian), look_at to view images, a couple for web search (web_search, read_thread, read_web_page), a few for subagents (Task, oracle), and a few for managing the agent TODOs.

Then, it has to provide a UI for the user where you can prompt it. A box you can type into, and then also visualization of the various model actions, like if it runs a command it'll show you the command and the output, or if the model edits a file it'll show you a diff.

But then to make these two basic tasks work well, it also has to provide a bunch of trickier / more complicated things. It's got a permissions system to allow or reject Bash commands, or ask the user for permission, and it can record and update the permissions over time. When the model runs a command it's got to start/stop processes. At the moment in Amp the model can't run background processes, but it sounds like a nice capability actually. Also right now there's this annoying thing that happens, where the model runs a command that enters an infinite loop, and then you have to manually stop it because the model can't; ideally Amp models could pause / resume process execution.

Also the user has to be able to interrupt the model (I do this a lot) or command execution. And one Amp feature that I absolutely love and have been using a lot is the $ key where I as the user can just execute a command and put the command and its output into the model context.

So putting all this together, the agent harness provides:

• A way to execute commands.
• A way for the user to prompt the AI model.
• A way to search for / edit files, view images, and search the web.
• Process control, permissions, interruption, but these are sorta intermittent and not fully baked I think.

When you put it this way, you realize that an agent harness is just a shell. Or, more precisely, it is:

• A shell for running commands and controlling processes
• Except it's "collaborative" (both the user and the model can use it) with a chat channel between them
• And the stuff that would require a curses app, like editing files, is offered natively, since models can't use curses apps right now
• Plus there's some random extra stuff like lightweight permissions, TODOs, and undo for file editing.

To me this is actually a pretty exciting vision of where agent harnesses could end up as the product matures. It could be a kind of "collaborative shell", where the user and the model are totally symmetrical, able to execute the same tools and commands, with the results (plus chat prompts) going into a shared context.

Ideally, both user and model could get the same UI treatment, maybe just distinguished by colors. I think this would really help teach people how to use the shell, a skill I see college students increasingly struggle to learn. Basically you could prompt the AI to do stuff, watch what it does, and then do the same stuff manually and have it work.

Plus, there are some nice UI affordances in existing agent harnesses that shells could adopt. Imagine if, when the user runs edit <filename>, the shell diffs the before-and-after to show a little inline diff. Naturally, edit <filename> would show a curses GUI for human users, while models would get a textual API. (For humans, it could dispatch to the user's preferred text editor, like Vim or Emacs, using the current $EDITOR method, but I increasingly find that the students I teach, at least initially, want a really simple Nano-style text editor.) Or think about the undo_edit feature, which I believe takes a reference to an earlier edit and reverses it. Humans could use that too, that would be nice!

Or consider permissions. With the AI model, it can be useful to restrict it to only touch files in the current directory, or to have it ask the user before doing dangerous stuff. But that's useful for humans too, just to prevent mistakes, especially for students. Naturally, over time, people gain confidence and don't want the hand-holding, but then I have also gained confidence in the models over time and also hold their hands less.

And one idea I've learned from the Zed guys is that collaborating with the model and collaborating with other humans is kinda similar and involves the same software substructure. I think that's true in this case too. A "collaborative shell" that involves two humans is kind of a niche product, but for doing deploys or debugging over Zoom or a few similar use cases it could be nice. If we get it as a byproduct of agent harnesses that would be great!

Imagine you start up your terminal, and it looks pretty much the same as today. The shell prompt is white—that's your color.¹ [¹ Maybe if there are multiple humans they each get a color.] You can type commands and they run, like normal; no AI yet. But you can also type a # at the start of the command (like a comment, get it?) and that'll prompt the AI. When the AI does stuff that stuff shows up in gold; naturally it can also send messages (to explain what it's doing), run commands, edit files, and so on. Besides the colors, your and the AI's actions show up identically.

The shell has all the normal process control features—suspend, background tasks, and so on. In fact, the model has access to them too; for example, maybe every time the model runs a command, it gives a timeout, after which the process is suspended and the model gets to examine output and decide whether to keep running or terminate; that would make infinite loops less of a problem.

It would also have a bunch of other nice shell features missing today. Commands would auto-complete. You'd get full line editing. Maybe it would be possible to run curses programs.² [² I don't think the model would be able to do this, though it does strike me as something models could probably do in the future without big breakthroughs.] You'd get command history and history search and whatever else. Globs, variables, etc. You could reuse an existing shell's programming language (Bash, Zsh) or make up your own (like fish did), though probably reuse is best.

Like today, the human should be able to interrupt the model (basically, both run asynchronously) or terminate its tasks. I'm not sure if the model should be able to control the human's processes; in reality I think it's safe to allow that, I would trust Claude with that power. Ideally, there's a permissions system underneath it all, where file access, command access, and process management can be configured for both the human and the model. By default, probably the human has all permissions and the model has far fewer, but you can imagine that for dangerous actions (like deploys or debugging on live servers) you'd restrict the human, or even maybe have the AI weigh in on potentially dangerous tasks.

Actually implementing this all would I think be doable. New shells do get made (fish shell, most prominently), as do agent harnesses. Neither is a big industrial project; Amp is I think about a dozen people. If you kept things simple (don't build your own text editor!) I bet one person could do it in a few months, or a larger group in somewhat less time.

Isn't this what the Warp terminal is? I don't know. I watched some videos of Warp and they seem much more focused on supervising AIs than directly using the terminal itself. But maybe this is part of the vision and I just don't get it. Honestly the videos have a bit too much marketing speak for me.

Basically, I think that with some evolution, agent harnesses might become the next generation of shells, which would improve non-AI use, make it easier to learn the shell, and make both human-AI and human-human collaboration better.