A customer support chatbot can take a real share of routine questions off your team. It can also become the widget everyone closes on sight. Which one you get is mostly settled before anyone writes code, in a handful of planning decisions about what the bot covers, where its answers come from and when it steps aside.
This post gives you those decisions as a checklist you can work through with your support lead in an afternoon.
Why most customer support chatbots get ignored
The numbers are humbling. In a Gartner survey published in 2023, only 8% of customers had used a chatbot in their most recent service interaction, and only a quarter of those said they would use it again. Results depended heavily on the kind of problem: just 17% of billing disputes were resolved by customers who used a chatbot along the way, against as many as 58% of returns and cancellations.
A year later, another Gartner survey found that 64% of customers would prefer companies didn't use AI for customer service at all. Their top worry was that it would get harder to reach a person.
Read together, the lesson is practical. Customers are happy with a bot that solves their problem quickly. What they dislike is one that blocks the way to someone who can.
Decision 1: Pick the questions, not the technology
Start with your inbox, not a vendor demo. Export the last three months of support tickets, emails and chat logs, and sort them into types. You will usually find that a small number of question types make up a large share of the volume.
Good first candidates share three traits:
The answer already exists in writing, or in a system you can look up.
The answer is the same for every customer in the same situation.
A wrong answer is annoying, not costly.
"Where is my order?", "What is your returns window?" and "Can I move my delivery slot?" usually pass. Billing disputes, complaints and anything involving a refund decision usually don't, which matches the Gartner resolution figures above.
Decision 2: Decide what the bot answers and what it hands over
Write this down as a simple table before the build starts. It becomes the bot's job description and the first thing you test.
Question type | Bot answers | Bot hands to a person |
|---|---|---|
Order status | Looks up the order and reports where it stands | Lost parcels, damaged goods, anything overdue past your threshold |
Policies and product facts | Answers from your published terms and product sheets, with a link | Exceptions, "can you make an exception for me" |
Bookings and delivery slots | Offers free slots from the live calendar and confirms changes | Requests outside the rules, repeated rescheduling |
Refunds and billing | Explains the process and collects the details | The decision itself, every time |
Complaints | Acknowledges and captures the facts | Always, quickly |
Decision 3: Choose the sources it may answer from
A customer support chatbot should answer only from material you have approved: your help articles, policies, product data and the live records in your order or booking system. If the answer isn't there, the bot should say so and offer a person, rather than improvise.
This matters legally as well as for trust. In February 2024 a Canadian tribunal held Air Canada liable for a refund policy its website chatbot described incorrectly, rejecting the argument that the bot was responsible for its own words. Whatever your bot says, your company said.
Two habits follow from that:
Show the source behind each answer, so customers and staff can check it.
Fix the document, not the bot. When an answer is wrong, the cause is usually an outdated or unclear page.
Decision 4: Design the handover first
The handover to a person is the part customers remember. Plan it with the same care as the answers.
Make "talk to a person" visible from the first message. Hiding it is the fastest way to lose trust.
Pass the whole conversation along. Nobody should have to repeat their order number.
Land handovers in the helpdesk your team already uses, not a new inbox someone has to remember to check.
Say what happens next and when. "A member of our team will reply by email within four business hours" beats a silent queue.
Set triggers: two failed attempts, an angry tone, certain keywords, or any request on the "always a person" list.
Decision 5: Connect it to your systems, or keep it small
A chatbot that can only repeat your FAQ page saves little. The questions that fill inboxes are about a specific order, booking or account, and answering them means a read connection to your ERP, shop, CRM or booking system.
Decide early which systems it may read, which actions it may take (moving a delivery slot, say), and which it must never take on its own. Changes that cost money or alter a contract should go to a person for approval.
If those connections aren't possible yet, launch with a narrower scope and say so on the page. A small bot that is right beats a broad one that guesses.
Decision 6: Set the tone and the limits
Write a one-page brief, the way you would for a new support hire:
How formal it should sound, and which words to avoid.
Which languages it should answer in.
That it introduces itself as an automated assistant. Customers should never wonder whether they are talking to a person.
Topics it declines politely: legal advice, medical advice, opinions on competitors.
What customer data it may show, and only after what check (an order number plus email, for example).
Decision 7: Agree how you will measure it
Pick measures before launch, and look at them weekly for the first few months.
Resolved without a person: conversations where the customer got an answer and didn't contact you again about it within a few days.
Handover rate and reasons: the reasons are your to-do list for new documents or connections.
Unanswered questions: questions the bot couldn't find a source for.
Customer rating after the conversation, kept to one question.
Running cost per conversation, so you can compare it with the cost of a ticket.
Be wary of "containment" alone, the share of chats that never reached a person. A high containment number can just mean customers gave up looking for a way to reach a person.
Run a pilot before you switch it on for everyone
Put the bot in front of your own support staff first. They know the awkward questions customers really ask, and they will find the gaps in a week. Then open it to a small share of website visitors, or to one product line, for a few weeks.
During the pilot, read a sample of full conversations every few days as well as the dashboard. You will spot answers that are technically correct but unhelpful, handovers that fired too late, and documents that need rewriting. Fix those, then widen the rollout.
Tell your support team what the bot is for before launch. When they see it as the thing that takes the repetitive questions off their plate, they help improve it. When they see it as a threat, the feedback dries up.
Your planning checklist
Three months of tickets exported and sorted by question type.
Three to five question types chosen for launch.
Answer-or-hand-over table agreed with your support lead.
Approved source documents listed, owners named, outdated pages fixed.
Handover path tested end to end in your helpdesk.
System connections and allowed actions written down.
One-page tone and limits brief signed off.
Measures and a weekly review slot in the calendar.
A pilot with staff and a small group of customers before full launch.
Where to start with a customer support chatbot
If you want help turning your inbox into a plan, our chatbot development team builds support bots that answer from your own documents, name the source behind each answer and hand the hard cases to your people. For the reading-and-checking work behind the bot, our AI automation service covers the rest. A free 5-day audit of your support team is a low-risk first step, or you can start with a 30-minute call with a founder.
