Conversation design is hard. Though not completely unlike designing for mobile and web interfaces, conversational experiences are still a unique problem area to solve for and require specific methodologies as part of the design process. The feedback loop between user interaction patterns and the underlying technology is much tighter in Conversational AI than for other kinds of UX or product design projects.

In conversation design, user research and usability testing should continuously feed into the design because human communication can be hard to predict and user expectations keep changing as the technology evolves. Additionally, and this might be more of a tangent: conversational products cannot simply “exist”— conversational products have to constantly prove their value for existing through successful user interactions and usage. The cost of emerging technology is high, and not all companies or teams can afford to support an interface their customers are not using.

Below, I’ve listed and described a few of the methodologies used in conversation design.

Communicative goals

Before starting any project, you should identify the goal or desired outcome of the conversation both in the perspective of the user and the conversational agent. For example, a business wants to create a chatbot that stores all store orders in a database to help digitally track all orders coming into the store. Their goal is to standardize the ordering process and, by extension, it’s their chatbot’s goal to capture all details pertaining to an order from a customer. On the customer side, they want the chatbot purchasing experience to be faster and smoother than it would be going to this store in person or calling a human agent over the phone.

It’s important to identify the motivations behind creating and interacting with this experience so that you can help keep the conversation on track to reaching these goals. Note: this is not a replacement for setting KPIs or ways to track the product-defined success of an experience. Instead, think of this method as a way to define a North Star of the experience, and as a way to identify any obstacles that might get in the way of continuing the conversation.

To keep all of these considerations top of mind, I highly recommend following the Conversation Design Canvas shown to me in the certification course previously offered by Voice Tech Global. Their approach takes the concept of journey mapping and applies it to the context of a conversation. Some questions to ask are: When and where will the conversation happen? Who initiates the conversation and why? Which personality traits will help relate to your users and best portray your brand?, among many others. Another great resource is the Conversation planning page on IBM’s site.

Identify any obstacles that might get in the way of continuing the conversation.

Utterance farming

Okay, don’t quote me on this name, because I don’t remember the actual name of this research method (I’ve heard “utterance capture”, but I like to call it “utterance farming”). What is it? Utterance farming helps you, the designer, inform your design early-on about the ways users are likely to interact with your conversational experience.

It can be as simple as briefly outlining the context and role of the conversational assistant to a research participant and asking them how they would ask the assistant for a certain thing. For example, a prompt could be: “Assume you’re talking to a calendar scheduling voice assistant, how would you schedule a new meeting?” and you’d listen to the different things they say. The more variations, the better. From there, you can start noting grammatical patterns or even, more broadly, common phrases you can group into different conversational topics, i.e. intents.

Pay close attention to the wording you use in your questioning because you may risk over-prompting the participant. Essentially, when you provide an explicit utterance example (e.g. “schedule a new meeting with Elaine at 5”), you may limit their perspective on what they can say and bias the feedback they give. The idea here is to get an early reading of user expectations, so the more unbiased you can get, the more prepared you’ll be for later stages in the design process.

Inform your design early-on about the ways users are likely to interact with your conversational experience.

Intent sorting

The jury’s still out for the “most optimal” way to go about creating intents and even *when* to do this part of the process (see Brielle’s twitter poll below), so I’m arbitrarily writing this section here.

Quick recap: intents help our bots interpret conversation by sorting different actions or ideas behind different “themes”. A great way to start recognizing different intents you want your conversational experience to support is by creating lists based on existing data your company or client might already have, things like: past call transcripts or an existing record of emails and help requests. From this data, you’ll be able to identify the patterns of the things people ask for and how they go about asking for it.

Even if you don’t have access to this data, you should still define the scope of the experience and which actions you’re able and willing to support. No conversational experience will have expertise about everything, so it’s better to make sure what do you support is a good experience for the user.

It’s also important to know much overlap and ambiguity your particular technology can handle. Generally speaking, natural language systems don’t support having the same utterance nested under more than 1 intent. The danger here is that if you write similar sounding utterances for multiple intents, your users might get routed to the wrong path. Another fact is that utterances are not always well formed, i.e. users don’t use perfect grammar all the time. For this, you should find out if your build platform offers autocorrect or other forms of utterance “deciphering”. As a designer, you should engage your engineering team early and find out the quirks and limitations of the system you’re working with.

Identify the patterns of the things people ask for and how they go about asking for it.

Sample scripts

Sample scripts are exactly what the name implies: a brief look into the back-and-forth conversation between a user and the conversational agent. It’s a low-effort way to materialize what you already know about your users and your conversational experience. Designer Jesús Martin says: “dialogs help us outline interactions rapidly, allowing us to validate how they sound and how they flow.” They’re useful for experimentation. They’re also useful to convey changes to your copy over time.

Here’s a quick explanation on the topic by Cathy Pearl:

Materialize what you already know about your users and your conversational experience.

Flow mapping

Flowcharts — we love to hate them in this field, but more important than the style or tool on which to create these “flows” is the information architecture of the experience itself. At this stage in the design, your aim should be to convey the core elements of the experience, along with details of the conversation pace, flexibility, and recovery. When people ask you, “is the conversation design ready?”— this is the deliverable they’re likely referring to.

Flow diagrams are an essential tool to display your conversational experience at a high-level, without getting bogged down by the details of the actual responses and prompts. A good starting point is mapping out the happy path of one intent. Sketch out each step in the conversation from start to finish. Look for areas in the conversation where things could go wrong or a user might go off script, and design the repair pathways that branch from the original happy path.

One of your goals while mapping should be to call out the areas in your design that require additional knowledge gathering or behind-the-scenes actions (think: API calls). Additionally, your flow should identify all possible user journeys and the conditions that apply to their unique journeys. Take for example: you’re designing an IVR for a hotel. Many hotel chains nowadays have loyalty programs so you might be asked to design a curated journey for members-only and another for general guests, but both groups of users call into the same phone number. You may want to clearly label the step in the flow where you sort loyalty members from guests, and visualize that difference on the diagram so your stakeholders understand which questions (like verifying membership) are members-specific and which are for other guests.

Convey the core elements of the experience, along with details of the conversation pace, flexibility, and recovery.

Wizard of Oz testing

Personally, WoZ testing is my favorite part of being a conversation designer and I don’t get to do it often enough. It’s harmless occlusion of fact— you place a user with what they believe is a fully-functioning conversational experience or prototype, but actually it’s you, behind-the-scenes, feeding the bot its lines. The adrenaline rush is real.

Wizard of Oz testing is a form of usability testing and can be as low or high production as needed. The point of this exercise is to pinpoint gaps in your design (did a user ask for a feature that wasn’t but should be available? did they try to interrupt the bot?) and to get explicit and implicit feedback on the dialogs (did they furrow their brow in confusion upon hearing a particular bot response? did they ask for a clarification before answering?) to improve your design through iteration.

Not every project needs WoZ testing and not all prototypes need to be high-fidelity. It’s very much possible to conduct voice WoZ testing over zoom, camera off, an easily accessible design reference with all possible dialogs, and your laptop’s voice accessibility settings. As with all testing, remember to go into this knowing that your learnings may not be a holistic representation of all of your users’ expectations.

Pinpoint gaps and get explicit and implicit feedback on the dialogs to improve your design.

Final thoughts

There is no golden or “best” process to follow in conversation design. Each project will come with its own nuances, resources, and requirements. I mentioned flowcharts, but sometimes while working closely with models you might find flowcharts aren’t needed and you’ll write guidelines instead. This post provides a quick snapshot of methodologies that are specific to CxD, but may exist in other design disciplines as well.

Ultimately, it’s your responsibility as a designer to choose the “how” and the “what” of the design process. Choose what works for you and your team! Don’t worry too much about following every single step in the design process. Great design is iterative and adaptive. Happy designing!

Elaine Anzaldo is a certified Conversation Designer and has worked in various roles within the voice tech industry for the past 3+ years at companies such as NLX, Apple, and SRI International. As a designer for both large voice assistants and the customer self-service industry, Elaine has created conversational artifacts for voice, chat, and multimodal interfaces. In her spare time, she is also an evangelist for Conversational AI technologies, writing articles to help aspiring Conversation Designers get into the field and volunteering with the Voice This! Podcast team.

LinkedIn | Twitter | Instagram | YouTube