Designing for probabilistic output means designing for the wrong answer as much as the right one — confidence, correction, fallback, and knowing when not to use a model at all. I've shipped this on an AI receptionist and a quoting platform, where being wrong in front of a customer costs real money.
You're adding AI to a product and the demo works but the real thing doesn't feel trustworthy. Users can't tell when the model is guessing. There's no obvious way to correct a wrong answer. Someone has asked what happens when it's confidently wrong in front of a customer and the room went quiet. Or the opposite problem: you've been told to add AI and nobody can name the task it would actually improve.
I start by finding where a model earns its place — synthesis, drafting, triage, retrieval across your own data — and where it just adds novelty and risk.
Then I design for the wrong answer as carefully as the right one. Confidence signalling, correction paths, graceful fallback, and clear boundaries around what the system will and won't attempt. This is the part most AI features skip, and it's the part users judge you on.
I prototype against the manual version it replaces and test whether it's actually better. If it isn't, I say so.
I've shipped this on an AI receptionist and a quoting platform, where being wrong in front of a paying customer costs real money — which concentrates the mind on failure states.
Three to six weeks depending on how much is already built. Works best embedded with the engineers rather than delivered as a specification.
Do you build the model?
No. I design the product around it and work with your engineers on what the model needs to return for the interface to work.
What if AI isn't the right answer?
Then that's the deliverable. I'd rather tell you a feature doesn't need a model than help you ship one that adds cost and unpredictability for no gain.
How do you handle hallucination in the interface?
By designing for it rather than hoping. Show the source, make correction cheap, and constrain the model to tasks where a wrong answer is recoverable.