I have worked on testing a chatbot for HR processes in our company where each of its components like from input, to the knowledge base, intelligence and reasoning, taking into consideration the infrastructure where the bot is hosted, as well as other premises like connectivity and voice communication is tested.
To test usability, I had created a list of possible user inputs, together with the chatbot’s expected answer as well as potential problems such as alternative spellings or misspelling to ensure it still produces the same correct outcome or ask for further clarification. It is necessary for a chatbot to understand when the user made a mistake or to get the frustration arising from not getting the answer they are looking for.