AI needs good engineering: what separates a demo from something you can run

Getting an AI to produce something impressive takes an afternoon. Getting it to produce the right thing every day, safely, at a predictable cost, while you sleep, is an engineering job.

That gap catches a lot of businesses out. AI adoption is racing ahead: the British Chambers of Commerce found 54% of UK firms actively adopting it this year, and Simply Business found that security and privacy were the biggest barrier for small businesses that hadn’t started. Those worries are reasonable. They’re also mostly answerable, because the fixes are the same engineering practices that make any business system dependable. The AI part is a small piece of a bigger system, and the bigger system is where projects succeed or fail.

Here’s what we’d insist on for any AI project, ours or anyone else’s. (Rory’s background is in cloud computing and back-end systems, so a lot of this is the unglamorous plumbing underneath the clever bit.)

1. Ground it in your data, and check the output

A language model will happily write a confident answer that isn’t true. The defence is architectural, not hopeful:

• Fetch facts from your systems. Prices, stock, account details and dates should come from the catalogue, the accounts or the database, then be handed to the model, not recalled from its memory.
• Validate before acting. If the model extracts an invoice total, check that the line items add up. If it classifies an email, check it against the categories you allow.
• Make “I don’t know” an acceptable answer. Route low-confidence cases to a person rather than guessing.

This is how our timber enquiry agent works: it gathers the real answers first, then writes the reply.

2. Keep a human in the loop where mistakes are expensive

Not every action deserves the same trust. Sort them:

• Safe to automate: low cost, easy to reverse, easy to spot (tagging an email, drafting a reply for review).
• Needs approval: customer-facing messages, anything involving money, anything hard to undo.
• Never fully automatic: decisions with legal, safety or significant financial consequences.

Start with approval on everything, watch the results, and loosen only where the evidence supports it.

3. Treat prompts and models like code

The instructions you give an AI are part of your system. They need the same care as software.

• Version control. Store prompts and settings in a repository so you can see what changed and roll it back.
• Test cases. Keep a set of real examples with known good answers and run them whenever you change anything. This is the AI equivalent of automated tests.
• Model changes are changes. Providers update and retire models. Pin versions where you can, and re-run your tests before switching.

Without this, a small tweak on a Tuesday quietly changes behaviour on Wednesday and nobody knows why.

4. Host it properly, in your own cloud accounts

Where the system runs matters as much as what it does.

• Your accounts, not ours. Infrastructure should sit in cloud accounts you own, so you control access, data and cost, with no lock-in to a consultant.
• Least privilege. Each part of the system gets only the permissions it needs. An enquiry agent that reads the catalogue shouldn’t be able to delete invoices.
• Secrets stay secret. API keys live in a secrets manager, never in code or an email.
• Infrastructure as code. Define the set-up in files you can review and recreate, rather than clicking through a console and hoping you remember.
• Environments. A separate place to test changes before they touch real customers.

These are well-understood practices in cloud computing. They apply to AI projects exactly as they apply to any other.

5. Protect customer data

Be able to answer, in plain English:

• What data does the AI see?
• Where is it processed and stored, and for how long?
• Is your data used to train anyone’s model? (Check the provider’s terms for the plan you’re on.)
• Who within the business can see inputs and outputs?

Send the model the minimum it needs. Strip or mask what it doesn’t. If you handle personal data, this is also a UK GDPR conversation, so involve whoever looks after data protection for you.

6. Log everything, and watch it

When something goes wrong (and eventually something will), you need to know what happened.

• Record each step: the input, what was fetched, what the model returned, what action followed.
• Alert on failures. A silent failure is worse than a loud one.
• Track the trends. How often do people override the AI? Which kinds of request get flagged? Those figures tell you where it’s working and where it isn’t.

Good logging also answers the awkward question “why did it say that to a customer?”

7. Control the cost

AI usage is typically billed by volume, and a loop or a spike can run up a bill quickly. Sensible limits help:

• caps and alerts on spend;
• sending routine work to smaller, cheaper models and keeping the big ones for hard cases;
• caching answers that don’t change;
• measuring cost per task, so you can compare it with the time saved.

If you can’t state what each enquiry or document costs to process, you can’t tell whether the project is paying back.

8. Plan for failure

Models go down. Providers change terms. Networks drop. A dependable system has a fallback: queue the work and retry, or hand it to a person, rather than failing silently or losing it. Ask “what happens when the AI part is unavailable?” before launch, not after.

9. Document it for the next person

If only one person understands the system, it’s a risk. Good documentation covers what the system does, what it connects to, how to deploy it, what the alerts mean and what to do when they fire. You should be able to hand it to another competent developer and have them carry on.

A short checklist before you sign anything

Ask any supplier, including us:

1. Where does our data go, and who can see it?
2. What does a human approve, and what runs on its own?
3. How do you test it, and what happens when the model changes?
4. Whose cloud accounts does it run in?
5. How will we know when something breaks?
6. What does it cost per transaction at our volume?
7. If you vanished tomorrow, could someone else maintain it?

If the answers are vague, the project is probably a demo.

Why this is cheaper than it sounds

None of this means a bloated project. The reliable version of a small AI tool is usually only slightly bigger than the flimsy one. It’s mostly the discipline of doing the ordinary things properly, once, at the start, which is far cheaper than fixing a customer-facing mistake or untangling a system nobody understands.

Talk to us

If you’re planning an AI project, or you’ve seen a demo and want to know what it would take to run for real, get in touch. We combine AI workflow design with solid cloud and back-end engineering, which is why the things we build are meant to keep running. Our Automate and Build services cover the work, and a discovery sprint is a low-risk first step. For help choosing a first project, see Your first AI project.