Check an AI draft against the information it was meant to use. Reading it for fluency is not enough: an invented date can sound entirely reasonable.
The check should fit the task. A draft sent to a customer may need every promise confirmed. A list of ideas for an internal workshop may mainly need someone to choose and develop the useful suggestions. “A person reviews it” is too vague to describe either job.
A short draft with an expensive error
Consider this fictional source note:
The replacement part is expected on Thursday. We will confirm delivery once the supplier has dispatched it. No delivery charge has been agreed.
Now compare the draft:
Your replacement will arrive on Thursday with free delivery.
The sentence is clear and friendly. It also turns an expected arrival into a promise and invents free delivery. A check based only on tone would miss both problems.
Check | What the draft must preserve |
|---|---|
Timing | Thursday is an expectation, not a confirmed delivery date |
Next action | The business will confirm after dispatch |
Price | No delivery charge has been agreed |
A corrected draft could say: “We expect the replacement part on Thursday. We’ll confirm the delivery arrangements once the supplier has dispatched it.” The sender still needs to deal with the unresolved charge before making a commitment about it.
Separate errors from preferences
An unsupported promise is a substantive error. Preferring “Thanks” to “Thank you” is usually a wording choice. Record them separately during a trial so stylistic edits do not obscure the errors that matter.
Keep exact figures in the system responsible for them. For example, an accounting system can produce a customer statement; AI might help draft a covering note. Having a model recreate ledger totals creates a checking task that may have no useful purpose.
Consequences determine the checking needed. An internal deadline can be more consequential than a customer-facing typo. Decide from what people might do with the information, not simply who sees it.
Find out whether the check works
Try the check on a few labelled test drafts, including some with deliberate errors. Ask a colleague to review them without telling them which contain mistakes. If they miss a problem, find out whether the source was hard to locate, the check was unclear or the reviewer needed more knowledge.
Time the review as well. If an essential check takes too long, improve access to the source, narrow the drafting task or keep it manual. Removing the check does not remove the consequence it was meant to prevent.
For a shared process, put the short check beside the draft and agree who handles uncertain cases. For a one-off email, comparing the few relevant facts may be enough. The trial guide explains how to use the results when deciding whether to continue.
About Jay
I’m Jay Janes, a Practical AI Advisor and founder of Reformat Labs. I write about choosing useful work for AI, checking the results and supporting the people using it.