Chatbot Testing: How to Get it Right in the First Go

Chatbots have forayed into the business arena with a bang. Enterprises have been leveraging chatbots to not only engage the end-users, but also study their behavior and habits. It is now established that a chatbot does much more than just messaging.

Residing within messaging platforms, chatbots are rapidly adding value to conversations. Chatbots now have a context-driven intelligence. The conversations aim at solving customer problems in the most convenient manner. The entire experience, thus, depends deeply on how tactfully the chatbot steers conversations to yield the most acceptable responses.

The conversation is the key skillset of a successful chatbot. An engaging bot is one that can understand the customer’s questions and respond promptly. But this is just the tip of the iceberg. Businesses need to thoroughly focus on enriching the user experience delivered by bots – as they emerge as one of the most promising digital touchpoints in a customer’s journey.

So, we need to establish what ground rules should be established for this new paradigm. Earlier, we talked about the qualities of the right chatbot development partner and gave a step-by-step approach to building a remarkable chatbot personality.

In this article, we will talk about the must-haves in your chatbot testing checklist. We will also guide you on how to go about these checks and balances.

SEE ALSO: What is ML governance?

1. Testing a Chatbot’s User Experience (UX)

The user experience of a chatbot consists of various aspects. It includes the visual appearance and the personality of the chatbot during conversations. While testing out your bot’s performance from the UX point of view, you must consider the following:

The Onboarding Process

Verify that onboarding is the first thing that the bot does when the user opens the chatbot window on your channel. For example, on Facebook, a greeting screen and greeting text should appear when the user clicks on the chatbot option. This allows the user to understand what the next step is. A ‘Get Started’ button can be a good idea for conversing for the first time.

It would be best if you established the ground rules before the conversations start. This feature gives the user a fair idea about what the bot will do.

Developer Tip: Consider this feature during the time of the bot’s development. Fill up on some onboarding tricks.

Tester Tip: Ensure onboarding in the chatbot. This will help you decide whether the user will stick around or go away. Take a look at the example below:

Using Generic Templates

The use of generic templates is a great way to ensure that the user feels connected while finding your bot on different channels. Generic templates help with better engagement. Your content cards must appear more or less the same across multiple channels to retain the user’s attention.

Developer Tip: Check JSON templates provided by channels and compare. Use those templates that allow you to make a content card with thumbnail, text, or title with a minimum of channel-specific properties.

Tester Tip: Make sure you check multiple channels to validate the content cards.

Balancing Out Text Length

It would help if you were mindful of the fact that it’s a conversation. Both responses should be concise, specific, and not a bunch of long strings of text. Text responses should not be broken. For example, channels keep a word limit as to how much can be included in a single reply. If it exceeds this limit, they break the text into two parts, which look hideous.

Developer Tip: Take into account a channel’s text limits for each message. Use paging when necessary.

Tester Tip: Verify that there is no wall of text or any broken text.

Ensuring Correct Link Navigation

The connector links included in a bot’s conversation must be checked thoroughly. The links should not be broken or redirect users to incorrect pages. This may otherwise leave the user with a terrible experience. They may end up never using your bot again.

Tester Tip: Ensure that the ‘back button’ immediately takes the user back to the bot interface

Including Feedback Options

A user would always love to share his experience, good or bad, with you. This will help you in precisely knowing the experience delivered to a specific user. Make sure a feedback point or “contact us” option is included with the bot. This small touchpoint within the bot can often be a savior for you.

Developer Tip: Feedback storage in the database should be part of your bot development.

Tester Tip: Verify if this feature is easily visible to the end-user. You must check it thoroughly with single-line and multiple-line inputs.

Users can put in “Very Likely” or “Likely”. “AskHaley” also recommends sharing the bot. If a user isn’t happy with the product, Ask Haley solicits open-ended feedback to learn more.

Testing Conversational Flow

This section deals with the context of the conversation. An appropriate conversation flow is one that talks tactfully and keeps the user engaged with relevant replies.

Ensuring a Problem-Solving Approach

Here, you need to examine if the bot is serving its core purpose. For example, if it’s a news bot, you need to test whether it is delivering the news properly. Or, if it is a conversational-commerce bot, whether it is able to search and proceed to checkout smoothly.

Checking Context Awareness

The bot needs to be intelligent enough to remember specific talking points. Consider this example:

BOT: Shall I arrange a callback

User: Ok / Yes / Sure

Here, you need to ensure that your bot is able to efficiently understand that the user wants a callback.

Developer Tip: Make sure you maintain a context stack in your conversation to keep track of important points during the conversation.

Tester Tip: Test by throwing off random words in the middle of a flow or leave the chat and then return to see if the bot is able to maintain the same flow.

Adding Alternative, Yet Valid, Responses to the Same Text

To break the monotony in a chatbot conversation, you must pep it up with exciting responses to important messages. Test whether expressions like” Hi, Hello….” yield different responses from the user. The bot could say, “Hello,” sometimes “Hi,” or “Hey.” This will add life to the conversation.

Developer Tip: Configure multiple responses to standard greetings or sign-off words.

Tester Tip: Test your bot by throwing off these familiar words in the middle of a conversation. See if the conversation breaks or if the bot smartly takes the conversation forward.

Including Pleasant Responses to Erroneous User Inputs

Users often tend to throw off the bot with challenging questions, assuming that it will magically understand everything. Here, your bot is at a real test. If it is unable to understand the user input, it must clearly state it. But saying this in a pleasant way is crucial to win the appreciation of the user. Make sure the bot handles such erroneous inputs in the friendliest manner. It may say:

“I am sorry I am unable to understand you!”

“Our Oompa Loompas are unable to understand….But here’s what they can do…..<show menu>”

Including Help Features

Every user is different and looks at the bot from a different perspective or expectation. Thus, a help center is a crucial feature within the bot. This will ensure the user does not drop out in the middle of the conversation. It is a good idea to introduce the help feature during the onboarding process.

3. Testing a Chatbot’s Intelligence Quotient

A chatbot has to exhibit a personality. It has to be witty, quick, and humorous to be called intelligent. So, this is one area where you need to make unfathomable efforts. Your bot must impress the user with its responses. This will involve the following aspects:

Sharpening Your Bot’s Memory

You need to test whether your bot remembers everything discussed earlier in the conversation and does not expect the user to type in the same inputs again.

For instance, if the conversation involves the bot asking the user’s name, you need to check if it remembers the name. You also need to check if it reflects where the user left off in the conversation.

This feature will also eventually help you expand the scope for your bot and deliver a more personalized experience to the user.

Developer Tip: Make sure to test state variables in the bot with multiple users. Check for variable-state safety with various conversations.

Tester Tip: List the data maintained in the bot and test it with multiple conversations to see if your bot remembers everything you shared with it.

Enhancing the Bot’s Ability to Validate the User’s Intent

Validating the user’s intent implies that the bot should understand exactly what the user wants to know. This relates to the moment where the bot can deduce what the user wants.

There are two kinds of bots – Rule-based and Artificial Intelligence (AI) based. Some bots do not have a” typing option” for users; they guide the user through the conversation while providing opportunities for users to click. Bots that involve a typing option need to validate the text. They need to check if the user has asked the same query multiple times and whether there are some typos or ambiguities.

Developer Tip: Either use buttons to present options or maintain a list of intents in conversation code to maintain the fluidity of a human conversation.

Tester Tip: List down the primary intents of the conversation flow and present them to the bot in multiple ways. This will help verify if the bot can recognize the objectives in conversation.

4. Chatbot Testing Response Time

Bots only involve conversational texts. So, response time is a crucial factor that drives user engagement. The responses have to be prompt.

Market surveys have shown that waiting during customer care calls has been one of the biggest frustrations customers have. The reason for this, mainly, is the waiting time. The customer will lose interest the moment he or she is asked to wait for the following response.

Hence, a bot, seen as a replacement for customer care representatives, cannot afford to keep the user waiting for its response.

5. Testing a Chatbot’s Security

A chatbot has access to sensitive data, as the responses need to be linked to customer information. This elevates security threats, too.

Masking Sensitive Data

Chatbots reside inside our social messaging platforms and have access to our sensitive data. This data includes the user’s personal information. A banking bot, for instance, may ask for sensitive info from the user. Imagine if the bot ends up exposing some of that critical piece of information during a future conversation. You cannot let this happen. Here’s what you need to ensure that info is secured:

• Create a mechanism to ensure controlled exposure of sensitive info
• Secure where sensitive data is stored

Testing for Outages

There are two aspects to this factor. One is killing the bot with vulnerabilities in the technologies used. The second is the old-school method: “spam the webhook with spoofed messages.”

Nowadays, most of the technologies are pretty mature, and most of the security glitches are fixed quickly. But due to the initial boom in bots, there was a mushrooming of bot libraries.

As I have worked with Facebook chatbots mostly, keep in mind the implementation of ”X-Hub-Signature” in any library you use. Make sure it is implemented the right way and not skipped. This can allow anyone to spoof the request and make your bot believe it’s from Facebook.

It is a Metasploit module for testing spoof of messages or a ‘way to kill your bot.’

Take a look at the following example from Facebook:

SEE ALSO: How AI Is Making Software Development Easier For Companies And Coders

6. Testing a Chatbot’s Multiple Channel Compatibility

Multiple channel compatibility is another crucial component when testing the bot. There are numerous messaging channels, and each has a unique way of technical implementation. So, if your bot is to live inside multiple channels, make sure it’s a happy living.

This factor allows you to validate and review the end user’s experience across multiple channels like Facebook, Telegram, Kik, etc.

Now, channels allow developers and testers to integrate the bot into a medium for testing purposes.

Check whether your images and templates (content cards) appear correct. Ensure the following:

• The button text appears properly
• Photos appear in the right place and don’t get cropped across different channels
• The ”bot responses” appear the same across multiple channels

Conclusion

In a nutshell, you should validate the above checkpoints as part of your regular bot-testing plan, and I am sure you will create something that will make the end-user happy. Regular testing of chatbots is the key to their success.