I must admit that monitoring of chatbots was something I was less familiar with within SAP Conversational AI. It seemed technical, full of stats and formulas, part of the generally boring part of maintenance of software.

But it seemed to be vitally important, so I made myself learn a little about it.

And I learned that this was vitally important in the development of a bot. Monitoring allowed you to understand how users were communicating with your bot, and let you make crucial changes because there was no one to know — no matter how well-designed your chatbot was — how users will react to it. As one person I talked with said, “When you deploy your chatbot, your development work is only just beginning.”

The new tutorial is called Improve Your Chatbot Accuracy by Monitoring User Activity, and delves into your “data set” — all of your intents and expressions and entities — and how to improve it based on what your users are doing.

Among the things you’ll learn:

• How to check the Log Feed, and then reassign utterances to the correct intent
• How to review Usage Metrics to see what intents, entities and skills are most and least used
• How to run tests with Training Analytics to see how well your chatbot discerns intents

But most important is how to check the Log Feed, not just to make your intents better, but to understand what your users really want your chatbot to do. This way you can see what additional use cases — that is, intents — you should add to your chatbot, and what functions you should try to automate via the chatbot.

We want to avoid the fallback skill at all costs:

And though I write it in the tutorial, I want to emphasize this idea:

You want your chatbot to be helpful to users, and not annoying. So your chatbot should, as much as possible, understand everything the user says. And then, the chatbot should automate at least 30-40% of the tasks the user asks about (the rest can be solved by referring the user to related documentation or phone number). It is better of more like 70% of tasks were automated.

And which tasks to automate?

The ones that most users want, and the ones that users use all the time — for example, like checking their balance or transferring money.

I want to especially thank Dhayabaran Jayaraman, who is one of the rare technical experts who is both extremely bright, understanding of users and customers needs, and capable of explaining it to someone not as bright as he. Thanks.

See all SAP Conversational AI tutorials

See all tutorial in the SAP Tutorial Navigator