A trial run on tidy examples can cost you the licence fee, a month and the goodwill of the person the job depends on. Your office manager tells you a tool could check each supplier invoice against its purchase order before payment, with your accounts assistant running it.

The usual trial goes like this. You run a dozen tidy invoices through it, the matches look good, and it goes live. Then your accounts assistant finds it passed an invoice charged above the agreed price, and quietly goes back to matching by hand.

The fix is to run the trial with that person, on real work, from the start. The steps run week by week and end in a clear go, change or stop decision.

Name the three people the trial needs

Start with the person who does the job today. In this case that’s your accounts assistant. She knows what a normal invoice looks like, which suppliers need different handling and which mistakes would turn into an overpayment. Without her, a tidy match can pass while the detail that matters is missing.

She runs the AI step as part of her real work and notes what she changes. She isn’t there to approve the technology or defend the trial. Her role is to show whether the check helps with the invoices she already has to clear.

Next, name the reviewer. Here it’s the office manager who approves each payment, comparing the tool’s result with the invoice and the purchase order and stopping anything that shouldn’t be paid. Protect that checking time rather than squeezing it between supplier calls. In a smaller business, the reviewer can be the person who does the job, as long as their checking time is protected too.

The third person owns the decision at the end. That’s probably you. You can approve a change to how invoices are cleared, give time for another trial or stop the work. You don’t need to read every match, but you do need to understand what was checked and what the corrections cost.

These three agree the standard before the trial starts. The method in agreeing what good looks like gives them a shared test, so the final decision rests on evidence rather than on whoever sounds most enthusiastic.

Run real invoices alongside the old way

Use the work that arrives, in the state it normally arrives. That means the real invoices and purchase orders your accounts assistant would have handled anyway, not a tidy sample you picked.

Keep the current process running alongside the trial until the new step has earned trust. The tool matches, and the accounts assistant still checks each invoice against the purchase order before it goes to the office manager. Nothing gets paid that the old process wouldn’t have caught.

Include the awkward invoices when they come in: the delivery split across two invoices, the supplier whose prices changed partway through an order. Don’t save only the clean ones because they’re easier to compare. Don’t invent hard cases to catch the tool out either. The right test material is the real mix the job produces.

The trial doesn’t suspend the business’s rules on supplier and bank data. If your approved setup can’t handle a record safely, that invoice stays in the old process and the reason goes in the trial record.

Before the first invoice goes through, collect the before position. Say your accounts assistant clears 40 invoices a week and spends 15 minutes on each. That’s 10 hours a week, and it’s the number the trial should bring down.

Set the end condition before you start

Run the trial long enough to see ordinary work repeat and a pattern form in the corrections. The calendar alone can’t set that. At 40 invoices a week, say, three or four weeks should show you the common invoice types.

So write the end condition down before you begin. The trial is ready for a decision when your accounts assistant has seen the common invoice types and the office manager can name the recurring corrections. The step also has to meet the agreed standard without special attention holding it together.

Book the decision meeting on the day the trial starts. That gives the work an end point and makes the owner responsible for what happens next. How long a first AI job takes shows where the trial sits between preparing your material and the later month of watching a live job.

Check corrections by cause every week

Once a week, sit down with the accounts assistant and the office manager for 30 minutes and read the corrections as evidence about the process. Group them by cause, not by the detail of each invoice.

A missing fact, such as an agreed price change, may point to the source material. A repeated misunderstanding may point to the instructions. A correct price matched to the wrong line of the purchase order may point to how the step sits inside the job.

Check the handovers too. When the tool lacks a purchase order, meets conflicting figures or gets an invoice outside the job, it should stop and pass it to the accounts assistant. A clean handover counts as correct. A confident guess doesn’t.

Ask the accounts assistant what happened before and after the AI step. The trial covers the whole step, including any copying and checking, not just the match.

End each weekly check by choosing the next fix and naming who owns it. Keep the list short enough to finish before the next review. You’re there to make the step work, not to collect observations for a report.

Fix the cause closest to each failure

Fix the cause nearest the failure. If the right information wasn’t available, repair the source or the route to it. If a rule was missing, add it to the instructions. If the tool passed an invoice it should have held, tighten the boundary or the stop rule.

Then run the failed invoice again and check the new result. That tells you whether the change worked, without waiting for the same kind of invoice to turn up by chance.

Don’t let anyone quietly redo every match and call the trial a success. A correction made over and over is part of the running cost. Either a process change removes it, or you accept it as a permanent checking step before the job goes live.

Change one thing at a time where you can, so the next review can tell which change helped. Keep the instructions and reference material with the trial record. This is also where the step starts to fit the working day. Making AI part of the work means giving it a trigger, a place, an owner and a route for when something goes wrong. The trial should prove those as well as the match.

Decide whether it goes live, changes or stops

At the decision meeting, put the original standard, the trial record and the open corrections in front of the owner. The answer takes one of three honest forms.

Go live when the common invoices meet the standard, the office manager’s checking time is acceptable and handovers reach the accounts assistant. Going live still comes with an owner and a review date. It doesn’t mean the step runs unwatched from day one.

Change the job and test again when the useful part is clear but the boundary, source material or checking step still needs work. Narrowing the step often works better than adding instructions. Matching only invoices with a single purchase order and leaving part deliveries to the accounts assistant is a perfectly good second trial.

Stop when the matches can’t meet the standard, the checking costs more than the step saves, or the setup can’t handle supplier data safely. I believe stopping is an honest result, not a failed one. Stopping leaves the old process in place and gives you a clear note of what would need to change before another attempt.

If it goes live, decide now where the saved hours go. Chasing suppliers for missing credit notes and keeping the purchase ledger up to date at month end are both jobs your accounts assistant would do better with more time. Don’t widen the job at this meeting. Prove the first step works in her hands, then decide whether the method belongs anywhere else.

Name the job, the person and the invoices

This week, spend 20 minutes with your accounts assistant. Write down the one step you want to test, in a sentence. Agree who will review, and pick the week the trial starts.

Then book two things before you leave the room: the 1-hour meeting to agree the standard, and the decision meeting at the end of the trial. Once the job, the person and the dates are written down, the rest of this playbook has something real to test.