Building an AI Chatbot That Actually Converts: What We Learned Shipping One
We shipped an AI chatbot on our own site before we ever built one for a client. That order matters, because the first version was a quiet failure and we would rather have learned that on our own traffic.
It answered questions beautifully. Visitors asked what we built, it explained. They asked about our stack, it explained. They asked what a project might cost, and it gave a careful, hedged, thoroughly unhelpful non-answer. Then they closed the tab.
Conversations went up. Enquiries did not move at all.
An AI chatbot for business is not a search box with better manners. It is a salesperson who never sleeps, and the difference between those two things is almost entirely in the design, not the model.
Chat volume is a vanity metric
The first thing we got wrong was the scoreboard. We tracked conversations started, messages exchanged, and questions answered. All three went up and none of them mattered.
The only numbers worth watching are the ones attached to outcomes:
- How many conversations ended with a contact request, a booked call, or an email address
- How many were handed to a human, and how many of those converted afterwards
- How many ended with the visitor asking something the bot could not answer
That last one is the most valuable thing your chatbot produces and almost nobody reads it. It is a continuously updating list of what your customers want to know and your website does not say. We rewrote two service pages purely from that log.
Ground it in your real content, or it will invent things
A model with no grounding will happily make up a feature you do not offer, a timeline you cannot meet, or a price you never quoted. On a marketing site, that is not a quirky hallucination — it is a promise a prospect will hold you to.
The fix is retrieval. Instead of relying on what the model absorbed in training, you feed it your actual pages, docs and service descriptions at question time, and instruct it to answer only from that material. Anthropic's guidance on reducing hallucinations is a good primer on why grounding beats prompt-engineering here.
Two practical details we learned the hard way:
Keep the source of truth in one place. Our chatbot reads the same content that renders our pages. When we update a service page, the bot updates with it. If you maintain a separate "chatbot knowledge base", it will drift out of date within a month, and nobody will notice until a prospect is misinformed.
Give it permission to not know. An assistant that says "I am not sure, let me get someone who is" reads as honest. An assistant that confidently invents an answer destroys trust the moment the visitor checks.
Decide what it should refuse
This was our biggest single improvement, and it is a product decision rather than a technical one.
Write down the things your chatbot must never do. Ours will not quote a firm price, will not commit to a delivery date, and will not give specific technical advice about a system it has not seen. Those are the three places where a wrong answer costs real money.
Refusing is not a failure state. It is the moment the conversation becomes valuable, because a well-designed refusal is a handoff:
I can give you the range we typically work in, but the honest answer depends on scope. Want me to set up a fifteen-minute call with someone who can give you a real number?
That sentence converts. The hedged non-answer we started with did not.
The handoff is where most chatbots fail
Here is the pattern we see on almost every business chatbot we have audited. The visitor asks something the bot cannot fully answer. The bot apologises. It suggests emailing support. The visitor does not email support. The conversation ends.
A lead arrived, identified themselves as interested, and was shown the door politely.
Three things fix it.
Hand off inside the conversation, not away from it. Do not send someone to a contact form in a new tab. Capture what you need where they already are — one field, ideally just an email or a phone number, with the conversation history attached so nobody has to repeat themselves.
Route to the channel they already use. We added a direct handoff to WhatsApp, and it outperformed the contact form immediately, because it costs the visitor nothing. They are already in that app. A form is a small chore; a message is not.
Make the timing explicit. "Someone will reply within a few hours" converts better than "we will be in touch", because vagueness reads as a brush-off.
If you are weighing up whether this is worth building properly for your own product, we are happy to talk through what it would take — including the cases where the honest answer is that a well-written FAQ would serve you better.
Keep the context window small on purpose
There is a strong temptation to stuff everything into the prompt: your whole site, the entire conversation, every past interaction. Resist it.
Long contexts cost more per message, respond more slowly, and — counterintuitively — often answer worse, because the genuinely relevant passage is buried among forty that are merely related.
What worked for us: retrieve only the few most relevant chunks for the current question, keep a short rolling summary of the conversation rather than the full transcript, and trim aggressively. The bot got faster, cheaper and more accurate at the same time.
Speed matters more than people expect. A reply that takes eight seconds feels broken even if it is brilliant. The Nielsen Norman Group's work on response time limits has held up for decades: past about ten seconds, attention is gone. Stream the response so the visitor sees words appearing immediately — perceived speed is most of real speed.
Guard the cost before it surprises you
An AI chatbot has a running cost that scales with how popular it gets, which is an unusual and slightly unpleasant property for a marketing feature.
Put limits in from day one:
- Rate limit per visitor, so one person cannot run up a bill out of curiosity or malice
- Cap the conversation length, with a graceful "let me connect you to someone" at the ceiling
- Use a smaller, cheaper model for simple routing decisions and save the capable one for real answers
- Log spend daily, not monthly
For most small and mid-size products this lands somewhere between $30 and $400 per month. Heavy chat volume can push it to $1,000 to $3,000 per month, which is entirely manageable if you planned for it and alarming if you did not.
What we would build first, if starting again
If we were doing it once more with what we know now, the order would be:
- Write the refusal rules before writing a single prompt. Know what it must not say.
- Wire retrieval to your live content, not a copy.
- Build the handoff path before polishing the answers. A mediocre bot with a great handoff beats a brilliant bot that dead-ends.
- Instrument outcomes from the first day, so you are never guessing whether it works.
- Only then spend time on tone and personality.
Most teams do this list backwards, starting with personality and ending with handoff. That is why so many business chatbots feel delightful and produce nothing.
Where a chatbot is the wrong answer
We should say this plainly, because it is the advice we most often give and it costs us work.
If your traffic is low, a chatbot will not fix that. Ten visitors a day will not generate enough conversations to learn anything, and you will have spent weeks on a feature that talks to nobody. Fix the traffic first.
If your questions are genuinely simple and finite — opening hours, shipping regions, refund policy — a clear page answers them better, loads instantly, costs nothing to run, and gets indexed by search engines. A model is a poor substitute for writing things down.
If your product is complex and highly regulated, the refusal list becomes so long that the assistant cannot say anything useful. In that case the value is in internal tooling for your support team, not a public-facing bot.
The situations where it genuinely pays are narrower than the marketing suggests: meaningful inbound traffic, a product that takes some explaining, and a sales process where getting to a human quickly is worth money. That describes a lot of software businesses. It does not describe all of them.
An assistant that knows when to stop being an assistant
The best thing our chatbot does is recognise the moment it is no longer the right tool, and pass the person to someone who is — quickly, without friction, and without making them start over.
That single behaviour turned a feature we were quietly embarrassed by into one that reliably produces conversations with real prospects. It is also, not coincidentally, what good customer service has always looked like.
If you are thinking about adding an AI chatbot for business to your product or your site, and you want it to produce leads rather than transcripts, book a free consultation call or write to us at nitheesh@playxoft.com. We will tell you honestly whether it is worth building for your situation — and if the answer is a better FAQ page, we will tell you that instead.
