Six rules for putting AI into customer support
The good version of AI in customer support is invisible: a customer asks a question at eleven at night, gets a straight answer, and gets on with their evening. The version that makes the news is the other one — the chatbot that lied, the assistant that gave away something it shouldn't — and after a couple of years of those stories it's easy to blame the technology.
Building this into StayUpfront has taught me that the gap between the invisible version and the horror story is a handful of design decisions, and almost all of them get made before any AI is involved.
I gave a talk on this at Twilio's "Conversations in the AI era" meetup in Manchester at the start of July, and these are the six rules I've ended up with.
1. Never hide the AI. Label every message it sends.
Customers don't mind AI, when it can help quickly resolve their problem or answer their question. What people hate is being deceived — an AI passed off as a person. That anger arrives at the moment they realise, and it sours the whole exchange, including the parts that went fine. Being upfront costs nothing. The other thing people can't forgive is being trapped in a room with it — which leads us to rule 2.
2. Build the exit. Count it.
An obvious way to reach a real human, always. Once you have that in place, the rate at which people take it is the best quality measure there is — every customer who gives up on the AI is grading it for you.
The catch is that the number is only honest while the exit stays easy. Bury the button, or make someone type "agent" three times, and the rate improves while the experience gets worse — which is how the industry ended up with chat widgets people fight their way past.
It's not just the customer that needs an exit, empower your agent with the same... let it identify for you when it needs to bring a human into the loop.
3. Only the caller's keys. Never the master set.
Earlier this year, attackers took over more than twenty thousand Instagram accounts. They didn't break anything — they simply asked. Meta had built an AI assistant for people locked out of their accounts, and it never checked that the email asking for a password reset actually belonged to the account. So attackers asked for reset links to be sent to their own addresses, and it obliged, until Meta pulled the tool.
That isn't a model being stupid. It's a bot that was handed the master keys. The lesson I've carried into my own build: instructions aren't locks — a model can be argued with, and eventually somebody wins the argument. The permissions have to be enforced by the system itself: when the assistant is helping one customer, that customer's records are the only thing it's given. Nothing else is within reach, no matter how smoothly you ask.
4. Keep it focused. One narrow job.
The pull is towards one assistant that does everything, and a capable model will cheerfully agree to try — I feel that pull constantly. What keeps me narrow is that narrow jobs are the ones I can test: "did it put this ticket in the right category" is a question with a right answer, and I can mark it. "Did it handle this ticket well" has no answer sheet.
You can ask a frontier model practically anything and it will happily oblige — this makes it tempting to give it lots of leeway. Nearly every time, it'll do a genuinely great job. But it's the other times you need to worry about. A narrow focus keeps the agent on the rails: less room to hallucinate, and a smaller, cheaper model will do — not nothing when the bills need paying. When the AI has that smaller scope it's also easier to notice the day it starts doing something it differently, and to quickly work out why.
5. Build a pipeline. Don't one-shot it.
Rather than hand a whole ticket over in one go, I chain small asks. The customer still sees a single reply box. Behind it: something reads the message and works out what kind of question it is; something checks whether it relates to a known problem; something searches the documentation; something drafts a reply. Then, before anything goes out: is this good enough to send, or does it go to a person, draft attached?
Several of those steps aren't AI at all — a lookup, a search. Ordinary software gives the same right answer every time, so the model is kept for the genuinely fuzzy parts.
6. Assume it'll fall over.
Anthropic's status page is a useful reality check. When I looked on 1st July, not one service showed 100% availability — claude.ai at 99.31%, the API at 99.55%, Claude Code at 99.41%, each measured over the preceding ninety days. An assistant sits on top of a service like one of those, plus everything built around it, so it can only do worse.
I'm building on the assumption those services will go down. More than one provider where that's practical, and a plain, non-AI fallback underneath everything: the form still takes the message, the ticket still lands in front of a person. When the model disappears, support goes back to the way it worked before any of this existed, rather than going dark.
None of this is theory for me — it's how I'm building Una, the assistant inside StayUpfront: labelled when it writes to a customer, scoped to whoever it's helping, and asking for input when it's not sure.
I hope these six rules are useful. Feel free to tell me where I've got them wrong — that's the more fun conversation 😉
– Rob