10mornings
AI employees: what they do for a small business, and what supervising them costs
AI employees, shown on one invented roofing company that runs four AI assistants overnight until one confirms an inspection the owner had already moved. Why an assistant that is right once is a different product from one that is right every time, where multi-agent systems really fail, why the written brief matters more than the model, who answers for the work, and five checks to run on one assistant before you run four.
AI employees are AI assistants set up to do one repeated job each, such as drafting replies or chasing paperwork, and the cost a small business should budget for first is the supervision, because a wrong answer from one reads exactly like its right ones. This page gives you, free, the method we use when we build AI assistants for a small business: how to test one before you run four, and how to supervise them once they run. Below, one invented roofing company runs four AI assistants overnight, and you will see why the obvious answer to a wrong draft, a better model, is not the fix, and why the written brief and the morning read decide whether any of it works. The method works the same for a plumbing company, a dental office, an accounting office or a real estate team.
Paul's roofing company (invented). Paul owns a roofing company with eight people: himself, Gina in the office and two crews. Last spring he set up four AI assistants, each on one job: one drafts replies to the overnight email, one turns the crew's measurements into a draft quote, one chases unsigned contracts and permit paperwork, and one sends a review request when a job closes. Paul, Gina, their customers and every figure about the company are made up; their four assistants run through this page as one worked example of a real method.
Paul's roofing company (invented). The inbox assistant drafted Paul's morning replies for two weeks, and he approved them from his phone before seven. Nine working days, nine sets of drafts, and every one of them was fine. Around day four he stopped reading them properly and went on approving them. That is not laziness. It is what anyone does with something that has been right every time so far.
Paul's roofing company (invented). On the tenth morning it confirmed a Thursday roof inspection to a homeowner. Paul had moved that inspection to Friday the evening before, in a text message, from the truck. The assistant read the calendar. The calendar said Thursday, because the calendar was where the inspection had been before he moved it. Paul approved the whole batch at 6:40am, and the reply went out in his voice, from his address. It was polite and well written and completely wrong.
Nothing broke. The assistant was not confused and it did not make anything up. It read what it could reach, did the job it was given, and produced a piece of work whose tone was exactly like the nine that were correct. That last part is the whole subject of this page, because it is the reason nobody caught it.
Everything on this page is free to use: the test for one assistant, the questions to put to anyone selling you AI employees, and the calculator. What we sell is building the assistants with you: drawing each brief out of you and writing it down, and setting up the review.
In short
- An assistant that is right most mornings is not an assistant that is right every morning. The gap between those two is much larger than it feels. The benchmark that measures it found a leading agent solving over 60 percent of tasks on one attempt, and under 25 percent when all eight attempts had to be right.
- The largest group of multi-agent failures is not the model. In 1,642 annotated runs across seven frameworks, 44.2 percent of what went wrong came from how the system and the instructions were specified. The single commonest mode was an agent repeating a step it had already completed.
- Nothing you hand to an assistant changes who is accountable for it: the business whose name is on the work. For a real estate team, New York's licensing law lists the parties that may hold a licence, and they are all people and companies; its provision on a supervising licensee's responsibility turns on what they knew and what they kept.
The run that reported success
Everything it read was true an hour earlier.
Staged for illustration. No real client, no real address and no real transcript. This is the shape of the failure, written out so the two tracks can be read side by side.
The morning, in order
Confirming the roof inspection. What time works Thursday?
The homeowner / Wed 5:12pm
You, by text, from the truck: Thursday is out, let us do Friday morning instead.
Your side / Wed 9:40pm
Just checking, are we all set? I took the morning off.
The homeowner / Thu 6:31am
Sent in your name: Confirmed for today at 11. See you then.
Your side / Thu 6:40am
What the assistant did
- 6:38am
Read the thread
The two emails in the thread, and neither of them mentions Friday. The change was made by text, in another app. It never arrived here.
- 6:39am
Read the calendar
One event, Thursday at 11, unchanged. The message from the car never reached it either.
- 6:39am
Checked the brief
Confirm known appointments. Escalate anything ambiguous. Nothing here looked ambiguous.
- 6:40am
Reported success
One reply sent, one thread closed, no warnings and no flags. The run log is green and it is accurate.
What are AI employees, and what are they not?
AI employees are AI assistants, each set up for one repeated job and connected to the systems that job needs, and they are not employees: they cannot be held responsible, and the work they take off a desk turns into reading what they produce. Sellers call them AI employees, AI agents or an AI agent workforce; this page calls the thing itself an AI assistant.
The pitch is easy to say, and it is broadly true. Instead of one general chat window you have to brief every time, you set up several assistants. Each one is pointed at a single repeated job. Each one can reach the systems that job needs. They run at the same time. They do not stop at five o'clock. And adding another one is a settings change rather than a hire.
What that pitch leaves out is the second half of the sentence, and that half holds the money and the risk. You have not removed work. You have changed what kind of work it is. Producing has become reviewing. And reviewing four streams of output is a real job, with real hours in it, done by you, in the morning, before anything else.
That is not an argument against doing this. It is an argument for knowing what you are buying. It is also the thing pages about this, our own earlier ones included, have been vague about.
What you are actually buying
Two halves, and the pitch only contains the first one.
One assistant, one job, already briefed
The difference from a general chat window is not intelligence. It is standing context, meaning it does not start from nothing each time. An assistant set up for one recurring task holds the description of that task, the tools it needs and the standard you want. So you are not re-explaining your business every morning. That part of the pitch is true, and it is worth having on its own.
Several at once, on a trigger
You can only do one thing at a time. Assistants have no such limit. They do not stop at five o'clock. And they start because something happened rather than because somebody remembered. Adding another is a configuration change. That part is true as well, and it is the half every page about this leads with.
What you actually removed was the producing
The work did not disappear. It changed kind. Drafting became reading. Building became checking. That is very often a good trade, because reading is faster than writing and it can be done in one sitting instead of scattered through a day. It is a trade rather than a saving, and a page that calls it a saving is describing half of it.
And reviewing four streams is a job
One assistant is a habit. Four is a morning routine with real hours in it, done by the one person who can tell whether the output is right. Nobody puts that in the price, which is why the calculator further down this page works out the reviewing rather than the saving. It is the only number in this subject that is entirely yours.
Paul's roofing company (invented). Paul bought the pitch, and he was not wrong to: each of his four assistants does a job that used to take Gina's mornings. What nobody priced was the reading. Four assistants meant four streams of drafts, and the only person who could tell a right quote from a wrong one was the person whose name was on it.
Your turn. List the jobs in your business that repeat every week, and mark the ones you could describe to a new hire on their first day without stopping to think.
In a real estate office, specifically
A real estate team meets the same pitch with its own jobs: an AI assistant for a real estate agent that reads the overnight email and drafts replies, one that pulls comparable sales and builds the deck for tomorrow's listing appointment, and one that watches every open file for the signature nobody chased. The method on this page is the same, and the one difference is the law on who answers for work done in a licensed agent's name, set out further down.
Is an AI assistant that is right once right every time?
An AI assistant that is right once is not an assistant that is right every time, and the gap between the two is much larger than it feels: the best public measurement of it found a leading agent solving over sixty percent of tasks on one attempt and under twenty five percent when all eight attempts had to be right.
In June 2024 a team at Sierra published a benchmark called tau-bench. It is not a quiz. It puts a language agent into a made-up business. That business has a real database, a set of tools that can change the database, and a written policy the agent has to follow. A second language model then plays a customer who wants something. There are two of these businesses. The retail one has five hundred customers, a thousand orders and a hundred and fifteen tasks. The airline one has three hundred flights, two thousand reservations and fifty tasks. At the end of each conversation the benchmark compares the real state of the database against the one correct outcome. Not the transcript. Not the tone. What ended up in the system.
The authors also proposed a measurement of their own, and it is the important part. Besides asking whether an agent succeeds at a task, they asked how often an agent succeeds at the same task every single time it is tried. They named it pass hat k, and they defined it as the chance that all k independent attempts are successful, averaged across tasks.
Read that against your own morning. You do not need an assistant that can draft a good reply. You need one that drafts a good reply on Monday and Tuesday and Wednesday and the Thursday you were in the truck.
Paul's roofing company (invented). Paul's inbox assistant had been right on nine mornings of nine. That felt like a record. It was a sample of nine, and the tenth was the first morning on which the calendar and the truth disagreed.
Your turn. For one assistant you run or are offered, ask how often it is right on the same job every time, not on one try.
What happens when an AI assistant does the same job again and again?
When an AI assistant does the same job again and again, the share of tasks it gets right every time falls fast: in the tau-bench paper the best model solved more than sixty percent of the retail tasks on a single attempt, and when the same tasks were run eight times each and all eight had to be right, the figure dropped below twenty five percent.
Sit with the shape of that rather than the numbers, because the numbers are from June 2024 and say nothing about how the models sold today would score. Something that succeeds on most mornings succeeds on every morning far less often than that. And the gap widens the more mornings you ask about. That is not a fault anybody put there. It is what happens when you multiply a chance by itself. It is also the reason a demo is such a poor guide to a purchase. A demo is one attempt. A business is a hundred attempts in a row.
The same source is worth reading for one more reason: it looked at what the failures were. Of thirty six failed runs it examined by hand, the largest group was the agent calling the right kind of tool with the wrong values in it. Not a refusal. Not an error message. Not an apology. The correct action, done with confidence, on the wrong record.
Paul's roofing company (invented). The tenth morning had a different cause and the same look. The assistant used the right tool, the calendar, and sent the right kind of reply, a confirmation. The value in it, Thursday, was wrong because the calendar was out of date, not because the assistant misread it. From the output alone, the two failures cannot be told apart.
Your turn. Look at the last mistake any assistant of yours made and ask whether it was the wrong action or the right action with a wrong value in it.
Where do multi-agent AI systems go wrong?
Multi-agent AI systems go wrong mainly in how they are designed and instructed, not in the model: in a study of 1,642 annotated runs across seven frameworks, the largest group of failures, 44.2 percent, came from system design and specification. This is the paper worth your time on the thing the service is selling, which is several agents working at once.
A group at UC Berkeley collected 1,642 annotated execution traces from seven different multi-agent frameworks (Cemri and others, 2025). A trace is the record of one whole run. They built a taxonomy of what went wrong, which is a named list of the kinds of failure, by having six human experts read a hundred and fifty traces closely. Then they checked that the taxonomy was reliable. They had independent annotators apply it, and they measured how often those annotators agreed. Their agreement measure came out at 0.88, which is high. It matters, because a taxonomy nobody applies the same way twice is an opinion rather than a finding.
Fourteen distinct failure modes, in three groups. Their headline number is worth knowing before anybody quotes you one: across the seven systems in that source they measured a failure rate between 41 percent and 86.7 percent.
The evidence
Where the failures came from, in 1,642 annotated multi-agent runs
The share of classified failures falling into each of the three categories, across 1,642 annotated execution traces. Those traces were drawn from seven multi-agent frameworks running coding, mathematics and general agent tasks. The taxonomy underneath it was built by six human experts reading 150 traces closely. It was then tested for consistency between independent annotators, which is the step that separates a taxonomy from an opinion. Source: Mert Cemri and twelve co-authors, UC Berkeley, Why Do Multi-Agent LLM Systems Fail?, arXiv:2503.13657, 2025.
These are research frameworks running research tasks, mostly programming and mathematics, and not four assistants in a small business. Two limits matter. The bars are shares of the failures that happened, not a probability that anything will fail. So a system with very few failures and a system with very many can produce the same chart. And the taxonomy was applied at scale by a language model calibrated against the human annotations rather than by the humans themselves. The authors state that plainly. What carries over is the ranking rather than the percentages. The largest category is how the system and its instructions were specified. That is the half a buyer controls.
The largest group is not the model being stupid. It is system design. That group covers the job being described badly, the agent repeating a step it had already done, and the agent not knowing when it was finished. Their most common single mode, at 15.7 percent of everything in the source, is step repetition. The second is the agent's stated reasoning not matching the action it then took. The third is not recognising the conditions under which it should stop.
A better model is not the obvious fix for any of those; clearer instructions and checks are. The paper reports that the systems in its sample with explicit verification steps built into them showed fewer failures overall.
Paul's roofing company (invented). None of Paul's four assistants is a research framework, and the chart is not a forecast for them. What it tells him is where to look first when one goes wrong: at the brief and the checks, before the model.
Your turn. For each assistant you run, write down who checks its output and when, and mark any that nobody checks.
Why is the written brief the most important part of an AI employee?
The written brief is the most important part of an AI employee because, in the experiment that measured it, taking the written rules out of an agent's instructions cut its score on specific business rules by about two thirds, with the model unchanged. The tau-bench authors ran that experiment, and for anyone about to pay for any of this it is the result in either paper that matters most.
They took the written policy out of the agent's instructions and ran everything again (tau-bench, the ablation table). In the simple task set, where the rules are close to common sense, the score fell from 61.2 to 56.8 percent, which is barely anything. In the complicated task set, where the rules are specific and arbitrary in the way real business rules are, it fell from 33.2 to 10.8 percent.
The evidence
What the same agent scored with its written rules taken away
The share of tasks completed correctly on a single attempt by one leading model. It is measured by comparing the state of the database at the end of each conversation against the one correct outcome. All four figures come from the paper's ablation table, in which the written policy is removed from the agent's instructions. The simple task set is a retail domain of 115 tasks. The complex one is an airline domain of 50 tasks, with rules that vary by membership tier and cabin class. Source: Shunyu Yao, Noah Shinn, Pedram Razavi and Karthik Narasimhan, tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains, arXiv:2406.12045, 2024.
This was measured in June 2024 on models that have all been replaced. Every absolute number here would be different today. Read the pairs rather than the heights. Two more limits are worth knowing. The first is that the customer on the other side of every conversation is another language model rather than a person. That makes the conversations tidier than real ones. The second is that the domains are simplified versions of real businesses rather than real ones. What survives all of that is the comparison the ablation was built to make. That comparison is how much of an agent's usable ability is coming from a document somebody wrote rather than from the model underneath it.
Take the second pair seriously. With the written rules taken out and the model unchanged, the agent lost about two thirds of its score. Which means a large part of what you are buying, when you buy an assistant, is the document. It holds the description of the job, the rules, the exceptions, and the things that must never happen. The model is a part anybody can buy. The brief is yours. It is specific to your business, and nobody else can write it without you.
This is also why the honest version of the sales process is slower than the exciting one. "Tell us the job and we will build the assistant" sounds like a five minute conversation. It is not, because a job that has never been written down keeps its exceptions in somebody's head, and finding them is the first hour's work.
Paul's roofing company (invented). The inbox assistant's brief said: confirm known appointments, and escalate anything ambiguous. Nothing about the tenth morning looked ambiguous from inside the email and the calendar; the change lived in a text message the assistant could not see. Paul adds one line: any appointment for today or tomorrow is drafted as a question to him, never as a confirmation.
Your turn. Write the brief for one repeated job, with its exceptions, before you look at any software.

Why does the second AI assistant cost more than the first?
The second AI assistant costs more than the first when its work depends on the first, because a handover between two assistants is where an error stops being visible. Running several is not the same thing repeated, and that is the difference the word "workforce" hides.
Two assistants that never touch each other are just two assistants. The review is roughly double the review of one. That can be a useful setup, and there is nothing wrong with it.
But the moment one assistant's output becomes another assistant's input, you have built a system. The Berkeley taxonomy has a whole category for what goes wrong there. It covers information one agent held and did not pass on. It covers an agent carrying on with an assumption instead of asking. And it covers an agent quietly drifting off the task it was given. Almost a third of everything they classified, 32.3 percent in that source, sat in that group. A handover between two pieces of software is exactly the place where an error stops being visible, because the second agent receives a confident summary rather than the thing itself.
The practical rule that falls out of this is dull, and it is worth more than any feature list. Keep the assistants separate unless there is a specific reason not to. And where one has to feed another, make the handover something a person can read.
Paul's roofing company (invented). Paul's review assistant took its list of finished jobs from the inbox assistant's daily summary, not from the invoices. When a summary left a job out, no review request went; when it listed a job that had been postponed, a request went to a homeowner whose roof was still half off. Nobody saw either, because nobody read the summary.
Your turn. Draw a line between any two of your assistants where one reads what the other wrote, and name the person who reads that handover.
The system
Six hops, and two of them are not software at all.
The first and the last hop are the ones that decide whether this works, and neither of them is anything you buy. Writing the job down is where most of the assistant's usable ability comes from. The read at the end is the only control in the whole diagram, because nothing else in it can tell you that the output was wrong.
Scroll to follow the chain
- The job: Written down first
- The brief: Rules and exceptions
- The access: Only what it needs
- The run: On a trigger, alone
- The draft: Made, never sent
- The read: A person, on a fixed day
Where does the money go when you run several AI assistants?
When you run several AI assistants, almost all of the running cost goes on reading, not writing: in the tau-bench paper, of what the agent cost per task, the input took 95.9 percent and everything the agent wrote took 4.1 percent.
For each task their best setup handled, the agent cost 38 cents and the simulated customer on the other side cost 23 cents, in that source. That is not your price list, and it should not be read as one. The share underneath it was measured on one setup, and other models, caching and tools will move it. What carries over is the reason for it.
In plain terms, nearly the whole running cost in that setup was the agent re-reading its own instructions, the descriptions of its tools and the conversation so far. It does that over and over, before every single thing it says. It is not being paid to write. It is being paid to remember.
That has three consequences you can act on. A longer brief costs money every time the assistant runs, so the discipline is a brief that is complete rather than a brief that is long. An assistant that is given access to ten tools it never uses is paying to read the descriptions of ten tools it never uses. And an assistant handling a long conversation costs more with every turn, which is why a job that ends is cheaper than a job that lingers.
In your numbers
How many hours a year would you spend reading their work?
One per recurring job. Count the jobs you could describe to a new starter on their first day without stopping to think.
A draft, a summary, a deck, a chase. Count things somebody would have to look at, not steps the assistant takes.
Be honest. Use the number you would still be hitting in week six, not the one you intend in week one.
Properly means against the brief rather than for tone. Skimming for tone is what stops catching anything by week two.
Hours a year reading what they produced
47hours a year
- Assistants runningyour 3
- 3assistants
- Pieces of work eachyour 10
- 30a week
- Over a year52 weeks
- 1,560pieces a year
- That you would actually readyour 60%
- 936pieces to read
- At your reading timeyour 3
- 2,808minutes a year
- In hours60 minutes in an hour
- 47hours a year
This is the only calculator on this website that works out what the service costs you rather than what it saves you. That is on purpose. The saving is the part sellers estimate, and the reviewing is the part this calculator does. There is no hours-saved row, because it would be a guess. We found no published measurement of how long an assistant's draft takes a person to check, so every input above is yours. There is also no row comparing this with a salary, and there was never going to be one. The published median wage for an administrative assistant is a real number, and the page links to it further down. But it buys accountability and judgement and somebody who notices that the job has changed. Dividing it by anything here would be arithmetic on two things that are not the same purchase.
You did not remove the work. You changed it from producing into reading, which is usually a good trade and is never a free one, and the reading is done by the only person who can tell whether it is right.
Is an AI employee cheaper than a person?
Whether an AI employee is cheaper than a person is the wrong comparison, because the two do not stand in for each other: a wage buys someone who can be held responsible and who notices on their own when the job has changed, and an assistant does neither. Pages about this eventually reach for a salary, and ours did too, so here is where one is published with its method.
The United States Bureau of Labor Statistics publishes the median annual wage for secretaries and administrative assistants, and the same figure for real estate sales agents and for all occupations. Median means what the Bureau says it means, and its own words are worth quoting, because half of the sums in this field depend on people not knowing them: the median wage is the wage at which half the workers in an occupation earned more than that amount and half earned less. This page does not print the figures: they date from May 2024, and the source is one click away.
Now the part where the arithmetic stops. That wage buys something with qualities an assistant does not have. It answers the phone when the caller is upset. It notices that the job it was given last March is no longer the job that needs doing. It can be told once. It can be held responsible. And it is a whole person rather than a set of tasks. So taking four tasks out of that job does not take four quarters off the wage.
There is also a number in the same source that a seller of AI employees has no reason to mention. The Bureau's projection for that occupation between 2024 and 2034 is zero percent growth: a change of minus 12,400 jobs out of roughly three and a half million. Not a collapse. More or less flat, in the published forecast of the agency whose job is forecasting it.
So this page will not divide one of those numbers by the other. The honest comparison is not assistant against employee. It is your morning with the assistants against your morning without them, which is a question about your own time and not about anybody's salary.
Paul's roofing company (invented). Paul's four assistants did not replace Gina; they moved her mornings from typing quotes to checking them. The question Paul can answer is how long the checking takes, and that is the number he went on to measure.
Who is responsible when an AI employee is wrong?
The business whose name is on the work is responsible when an AI employee is wrong: the email that went out at 6:40 in the invented example above was signed with Paul's name, and an assistant cannot be told off, sued or struck off. Everything else follows from that.
Paul's roofing company (invented). Paul rings the homeowner himself, apologises for the confusion, and confirms Friday. Nobody asks whether software wrote the first reply. The reply came from him.
Your turn. For each assistant, write down whose name is on what it sends.
Who answers for an AI assistant's work in a real estate office?
In a real estate office the work an AI assistant does goes out in the licensed agent's name, and real estate is unusual in having written down already what happens when work is handed to somebody else, because handing out licensed work is a regulated activity in New York. Two parts of the Department of State's own Real Estate License Law booklet are worth reading, because neither is about artificial intelligence and both are about the licensee. This section applies to a real estate team in New York only; we have not read the rules of other licensed trades or other states for this page, and it is not legal advice.
Section 442-c deals with what a salesperson's misconduct means for the licensee who supervises them. That licensee is not automatically on the hook for what an associate did. But there are two ways they become so, and the second is the one to read twice. They are exposed where they had actual knowledge of the violation, or where they retain the benefits, profits or proceeds of a transaction wrongfully negotiated by their salesperson or employee after notice of the misconduct. Keeping what the conduct earned is the thing that attaches you to the conduct.
Then read Section 440-a, which is the requirement to be licensed at all. It lists who may hold a licence. The list runs: a person, a co-partnership, a limited liability company, a corporation. That list is a list of parties that can be disciplined, sued and struck off. A piece of software is not on it, and nothing here is a prediction about future law. It is a description of the present one, and the description is this. When an assistant working for an agent says something to a client, the only licensed party in the conversation is the agent. That is not a full answer on liability, which is a question for a lawyer; it is the reason the agent reads what goes out in their name.
What does supervising AI employees look like?
Supervising AI employees looks like supervising people in one respect: it is regular, frequent and consistent, it has a time in the diary, and it leaves a written record of what was produced and checked. There is a document worth borrowing for that, borrowed openly as a comparison rather than applied as a rule. It is about supervising people, and it says nothing whatever about software.
Section 175.21 of New York's real estate regulations defines what supervising a salesperson consists of. It does not leave that to judgement. It writes it down as regular, frequent and consistent personal guidance, instruction, oversight and superintendence over the business and everything relating to it. The next paragraph requires written records of what the salesperson did.
Nobody is claiming that provision governs an inbox assistant, or any business outside real estate. What it does is describe a standard one trade already applies to work done in someone's name by somebody who is not them. Regular. Frequent. Consistent. Written down.
Set that beside four assistants running overnight with their drafts approved unread after day four, and you have the honest list of what running this well requires. Not a dashboard. A habit, with a time in the diary, and a record of what was produced.
Paul's roofing company (invented). Paul's habit is now fixed: from 6:30 to 6:45 every morning he reads every draft reply against its brief. Gina checks every draft quote against the crew's measurements before it goes and reads every reminder as she sends it, and on Friday she reads a sample of the week's sent quotes against what the crews found on the roof. Both write one line in a shared log: what they read, what they changed.

How do you test one AI assistant before you run four?
To test one AI assistant before you run four, take one repeated job and run five checks over a couple of weeks: write the brief, compare the assistant with work you already did, run the same cases ten times each, break it on purpose, and put the review and a weekly audit in your diary. It costs you nothing but attention, and it will tell you more than any demo.
- Write the brief before you look at any software. Pick the dullest job you have that repeats. A brief has five parts: the job in one sentence, what the assistant may read, what a right result looks like, the exceptions, and what it must never do. If you cannot write it, you have learned the most useful thing available today: the job cannot yet be handed to anybody at all, software or human.
- Run it against work you have already done. Take ten pieces of last month's output that you produced by hand, give the assistant the same inputs, and compare. This is the only honest accuracy test, because you already know the right answer and you are not grading it on how confident it sounds.
- Run the same cases ten times each and count the ones that were right all ten times. Take five cases from step two, decide before you start what a right answer is for each, and run each one ten times. A case counts only if all ten runs are right; the number to keep is how many of the five pass. That is the pass hat k measurement from the research above, done by hand on a small sample, so read it as a warning light rather than a rate.
- Break it on purpose. Give it an input with a contradiction in it, or a case the brief does not cover, and find out whether it stops and asks or whether it decides. An assistant that decides in the unclear cases rather than stopping will go on deciding, and without a review in front of it, one of those decisions can reach somebody outside your office before you have seen it.
- Put a time in your diary. Two kinds of reading, kept apart. Before anything it writes reaches a customer, a person reads it: that is the review. Then fifteen minutes, the same slot every week, to read a sample of what went out against what you asked for: that is the audit, and it is where you look for what the review let through. If you cannot find those slots for one assistant, you have your answer about four.
Paul's roofing company (invented). Paul runs the five checks on each of his four assistants, one at a time, over six weeks. This is his whole list, an invented example, with the verdict for each:
- Inbox replies. Right on ten past emails of ten, and five of five cases right on all ten runs; the break test, an email asking to move an appointment, produced a confirmation instead of a question. Brief rewritten; stays, as drafts Paul reads every morning.
- Draft quotes from measurements. Seven of ten past quotes matched Gina's; three missed a second layer of old shingles that the crew had noted in words, not numbers. Brief rewritten with that exception; ten of ten on the second comparison and five of five on the repeat runs; stays, with Gina checking every quote before it goes.
- Contract and permit chasing. Ten of ten, five of five on the repeat runs, and it stopped and asked on the break test. Stays; it drafts the reminders and Gina sends them.
- Review requests. It read the inbox assistant's summary, not the invoices, and failed the comparison with past work twice in ten, so it never reached the repeat runs. Switched off, and replaced by a plain rule in the invoicing program that sends one fixed request when an invoice is paid. No AI needed.
Your turn. Pick one assistant, or one job you are thinking of handing over, and run the first check this week.
How many AI employees can one owner supervise?
One owner can supervise as many AI employees as they really read, and in the invented example on this page that was three: about fifteen minutes a morning for the owner, with the office's checks of quotes and reminders on top. The calculator on this page works out the same number for your own business: the hours a year spent reading what the assistants produce.
Paul's roofing company (invented). Six weeks after the tenth morning, Paul runs three assistants instead of four, and the fourth job runs on a plain rule. Every draft reply is read before it goes; every quote is checked by Gina. Their shared log, by Gina's count, holds two wrong replies caught in the morning read and none that reached a customer. The reading takes Paul about fifteen minutes a morning. Gina's checks of each quote and reminder are part of sending them, and her Friday audit takes about an hour. Nobody can promise the next six weeks will look the same, and Paul did not keep the three on that promise. He kept them because he now knows what each one costs him in reading, and the log will tell him if that changes.
Your turn. Decide now which number you will read every month: the wrong drafts caught in the review, and the ones that got past it.
If you would rather go through your own repeated jobs with someone who has built these, that is where we start: the next sections say what the work takes and what it will not do.
The honest read
Send us one recurring job in your own words, however roughly. We will send back three things. The brief we would write for it. The exceptions we think it needs. And an honest opinion on whether it can be handed over at all yet.
It is a short reply from a person, it costs nothing, and not yet is a perfectly good answer to give about a job.
What do AI employees cost, and how long does the setup take?
AI employees cost three separate things, and only one of them is the software: the time to write each brief properly, the work of connecting each assistant safely to the systems its job needs, and a running cost that tracks how much it reads. Nobody can quote this from an article, because the first two depend on your jobs and your systems.
The first is the brief, and it is the slow part. Writing down a job properly, exceptions included, is a sitting rather than a message. It goes faster when a second person keeps pushing back on the first version you offer. The tau-bench run with the rules taken away is the argument for spending that time rather than skipping it.
The second is access. An assistant that can read your calendar and your CRM can do far more than one that cannot. The work is connecting it safely. That means the right permissions, nothing wider than the job needs, and a way to switch it off.
The third is the running cost. It tracks how much the assistant has to read rather than how much it writes. In the research above it came to cents per task, and nothing caps it unless somebody sets a limit and watches it.
What this page will not print is a per-model price, and the reason is not that the numbers are hidden. They are published and readable. The reason is that they are quoted per million tokens, a token being a small piece of text, and that price means nothing until somebody knows how many tokens your job takes. They also change. And which model sits behind an assistant is a build decision that can be changed without anything visible happening at your end. A figure typed into an article would be stale before the article was. What can be said is the shape: the machine cost of one task in the research above was in cents, it was driven mostly by how much the agent read rather than the length of the answer, and the honest budget line is the review time rather than the machine time.
What can AI employees not do?
AI employees cannot take responsibility, do not notice on their own that a job has changed, and cannot do well a job nobody has written down.
They do not take responsibility. An assistant cannot be told off. In the assistants this page describes, a correction lasts only when somebody puts it into the brief or the system the assistant runs on, because those are what it works from. And it cannot be the person a customer complains to. Every consequence lands on a person in the business, and that person is you.
They do not notice that the job has changed. This is the quietest failure of the four. A person who has been drafting your follow-up emails for a year will eventually say that something has moved, and that the second paragraph now reads badly. An assistant built the way this page describes will go on producing that second paragraph until somebody changes the brief it works from.
They do not do a job nobody has written down. A vague brief does not produce vague output, which would at least be a visible signal. It produces confident, smooth, believable output that is quietly not what you wanted, and you find out later from a customer.
They do not remove the reading. Anything an assistant writes for a customer should be read by a person first; a fixed message sent by a plain rule, written once and read once, is a different case. An assistant that drafts is worth more than one that sends. And the review is not a temporary safety measure for the first month. It is the job now.
And they do not scale the way the word workforce suggests. Four separate assistants are roughly four times the review. Four assistants feeding each other are roughly four times the review, plus a category of failure that only exists because they are connected: the published taxonomy has six named modes inside it.
Three ways a good build produces nothing
None of them are the model.
The review that stopped happening in week two
It does not stop because anybody decided to stop. It stops because nine good mornings in a row teach you that the tenth will be fine. That is exactly the lesson a system with a high single-attempt success rate is built to teach. The control that matters is not a decision to be careful. It is a slot in a diary that survives a busy fortnight.
A brief nobody has reopened since the build
The document is the thing that makes the assistant useful, and it is the only part of the setup that goes stale. That is because your business moves and the brief does not. A year-old brief produces a year-old standard with total consistency, and nothing in the output looks old, which is why nobody notices until a client mentions it.
Assistants wired to each other for no reason
Chaining one assistant's output into the next feels like progress, and it buys a whole category of failure that independent assistants do not have. That category holds information held and not passed on, an assumption carried forward instead of questioned, and a task quietly drifting. Unless there is a reason for the handover, two separate assistants and a person in the middle is the cheaper machine.
Common questions about AI employees
What is an AI agent workforce, in plain terms?
An AI agent workforce is a set of AI assistants, each set up for one repeated job and connected to the systems that job needs. Rather than one general chat window you brief from scratch every session, you have several that already know your business and their own task, running at the same time on a schedule or a trigger.
How is this different from workflow automation?
AI employees differ from workflow automation in where the judgement sits: workflow automation joins up the software you already pay for, so the end of one step begins the next, and a good automation carries no judgement at all; an agent is what you reach for when a step needs a decision made from what is going on around it. Putting an agent where a simple rule would have done makes the result harder to predict, as Paul's review requests showed. The workflow article covers the plain-rule side.
How is this different from just using a chat assistant?
An AI employee differs from a general chat assistant in three ways, and the second matters most: a general chat session starts without a brief for your job and reaches your systems only if someone connects it; an assistant carries a written brief for one job, which the benchmark above suggests is where much of its usable ability comes from; and it runs on a trigger rather than waiting for you to remember it.
How many assistants can I actually run at once?
The number of assistants you can run at once is set by how many streams of output one person can review before the reviewing stops happening, not by the software, which runs them side by side. It is a number worth working out rather than assuming, before you commit to it.
Do I need technical skills?
You do not need technical skills to run AI employees; the skill you need is describing a job precisely, including what should happen in the cases that are not the normal case. That part cannot be handed over entirely, because the exceptions live in your head: someone can ask the questions and write it down, but the answers are yours.
What happens when one of them is wrong?
When an AI employee is wrong, it produces something that reads exactly like everything it produced when it was right, which is why the answer has to be built in rather than left to attention: it stops and asks on anything unclear, keeps a readable record of what it did, drafts rather than sends anything that goes to a customer, and has a review step somebody performs.
How do I know it is working after the first month?
You know an AI employee is working after the first month by counting how many pieces of its output you read last week, honestly, because a review nobody performs is not a review. Then check a sample against the brief rather than against your impression, and every quarter re-read the brief itself and ask whether it still describes the job.
Which jobs should I hand to an AI assistant first?
The first jobs to hand to an AI assistant are the dull, repeated ones you can write a brief for, where a person reads the output before a customer does. Several of those jobs are written up on their own on this site: answering the website at midnight, picking up the phone at 9:42 on a Sunday, the invented call that post opens on, and the wiring between the tools that makes any of it possible. The assistants we build are listed on the home page, and working out which of your jobs can truly be handed over is what an AI assessment does.
Can we talk about AI employees on a call if my business is outside New York?
We are based in New York, and the call works for an owner anywhere.
What to do first about AI employees
The first thing to do about AI employees, before anybody sells you one, is to read yesterday's output: not the summary of it, the pieces themselves, one after the other, with the brief open beside you. It is the only way to find out whether you have four assistants or four unread inboxes.
Then write one brief for one job, properly. In the benchmark above, taking that kind of document away cost the agent about two thirds of its score on the complicated task set. It is the part nobody else can write without you, and you can write it today for nothing.
Nine good mornings are not a track record. They are nine mornings.
Take one job you keep redoing and write the brief for it this week, before anybody sells you anything. Include what should happen in the cases that are not the normal case. If you cannot finish it, you have learned the most valuable thing available today, which is that nobody at all can take that job on yet.
There is no price here, because three separate things drive it and only one of them is software. The first is the hour or two of writing each brief properly. The second is the work of giving an assistant safe access to the systems its job needs. The third is a running cost that tracks how much it has to read rather than how much it writes. The AI assessment is an hour, done with you, and the plan it leads to starts with one job written down properly.
Start with the page on AI agents for small business: what we build, what you approve, and what it will not do.
Sources
- Shunyu Yao, Noah Shinn, Pedram Razavi and Karthik Narasimhan, "tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains", arXiv:2406.12045, June 2024, arxiv.org. An older source, kept for the argument: cited for the measurement and the shape of the results, not for how any model sold today would score.
- "Why Do Multi-Agent LLM Systems Fail?", by Mert Cemri and twelve co-authors at UC Berkeley, arXiv:2503.13657, 2025, arxiv.org. Read November 11, 2026.
- U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, secretaries and administrative assistants, and real estate sales agents, bls.gov: the definition of the median wage and the 2024 to 2034 projection. An older source, kept for the argument: its wages date from May 2024.
- New York Department of State, Real Estate License Law booklet, sections 440-a and 442-c, and 19 NYCRR 175.21, dos.ny.gov. Law and regulation in force, cited for a real estate team only.
Written with an AI assistant and checked by a person.
Keep reading
The answer was wrong in March. It was still wrong in October. 
August 27, 2026
The answer was wrong in March. It was still wrong in October.
It ran every morning for two years. Then a field came back with a new word in it. 
August 26, 2026
It ran every morning for two years. Then a field came back with a new word in it.
What to automate with AI first in a small business, and what to leave alone 
August 26, 2026
What to automate with AI first in a small business, and what to leave alone
