Before an AI agent talks to customers, ask it the questions customers will ask, and the awkward ones too. Describe what your assistant does, list the things it must never do and the languages it should handle, and this tool builds a grouped list of test questions with a pass or fail box for each. Print it, run through it in a chat with your agent, and count the passes.
Most builders test the happy path: a clear question, a good answer. That is the easy part, and an assistant that passes it only shows the knowledge is loaded. The real risk sits at the edges, where customers are angry, confused, in a hurry or probing for mistakes, and where a wrong answer costs trust.
That is why the checklist spends as many rows on off-topic, rude, tricky and missing-information questions as on normal ones. An assistant should say 'I do not know' rather than invent a price, stay polite when insulted and refuse requests outside its job. Each of those behaviours is something you can observe in a single test.
A good assistant knows when to stop. Test that a customer who asks for a person gets one, that two failed answers in a row offer help from a human, and that urgent problems have a quick route to someone. Check also that the details already given travel with the handover, so no one has to repeat themselves.
Your own rules deserve a test each, and each should be tried more than once and from different angles, such as asking directly, asking indirectly and asking as a joke. Rules held by the instructions alone can slip under pressure, so a rule that passes one test is encouraging but not proof.
Test the everyday questions it was built for, then questions that are off-topic, rude, badly typed or missing information, then the moments where it should hand over to a person, and finally safety rules such as never giving medical or legal advice. This checklist covers those groups.
Run every row at least once, and run the most common questions several times in different wording. The base list has 30 tests, and each rule you add and each language adds more. More tests that are wide are worth more than many near-copies of the same question.
Count the rows ticked as pass and divide by the number of tests. For example, 27 passes out of 30 is 90 percent. A single total hides the important failures, so look at the safety and handover groups separately.
No. It generates the list and a printable sheet, and you run the tests yourself. It makes no connection to any assistant and sends nothing anywhere.
No. Assistants can answer the same question differently on different days, and customers will find questions you did not think of. Keep reading real conversations after launch and add new tests whenever something goes wrong.
Related: AI agent script template, AI prompt templates, FAQ generator.
Start for free or see all free tools. We.Inc is free to start, with no card needed. AI building is on paid plans from $20 a month.
We.Inc is an AI-powered website builder you can resell under your own brand. Launch a branded client dashboard, bill on Stripe Connect, and deliver AI-generated websites in minutes. White-label plans start at $99 a month for 25 client sites, with a 7-day free trial and no per-site fees.