The Conversational Experience Playbook: Part 2 – Design And Build

In part 1 of the playbook, we covered research areas required for conversational experiences and established frameworks for defining an experience. Once you have a clear definition of your experience, the next step is researching the right platform that fits your needs.

The current state of conversational platforms is exceptionally diverse but oversaturated in many ways. Finding the right tools can feel overwhelming, mainly because so many technologies appear to do the same thing. Thankfully there are many resources to get you started:

With a list of potential solutions, the first step to identifying your tools/technologies is understanding what functionality you will need short and long term. It’s best to think of functionality in three areas:

Chat Experience

Looking at your conversational use cases, identify what functionality you will need to support those use cases within a chat experience.

Chat Flow Design: What level of control will you need when creating chat flows? Will you be mapping against intents, which is more complicated but provides richer and more varying conversations? Or are you taking a more simple chat-flows approach that is easier to design and implement, but limits overall conversational variance? How much will you be leaning on NLP/NLU or other AI technology? This can be difficult to determine without having mapping your chat flows. It’s not uncommon to do some level of chat design before understanding what technology is right for your needs.
Interactivity and Rich Messaging: What type of interactivity do you need from your experience? In chatbots and text services, this can include serving users interactive content cards, linking users to other pages of a website, or even completing tasks within the chat. In voice experiences, this can include retrieving information from a data source, automating triggered messages, or completing specific tasks through voice prompts.

Language Capabilities

When determining if you need to leverage natural language processing or AI and machine learning, a good rule of thumb is to examine what you wish to return to users:

Existing Content & Information

If you are returning content to users from existing data sources such as knowledgebases, FAQs, product catalogs, or other structured data sets, you can use relatively minimal language processing in your experience. When pulling from these types of data sets, you can leverage search-like methods such as keyword and tag matching in place of analyzing entire sentences or analyzing multi-message contexts. For example:

A banking customer may ask how to look up their account balance in multiple ways; “How do I check my account balance?”; “Where do I find my balance?”; “How do I check how much I have?” Those are all very different prompts, but the likely returned data and response come from existing content and data sets like a help center. Because of this, the chat technology only needs to key in on select words like how, where, check, account, balance, to find the appropriate content and information from the data set.

Compiled Content & Information

If you are returning content from real-time sources or compiling data from multiple sources into a single response, you will likely need moderate or advanced language processing levels in your experience.

For example, what if that same banking customer requests their actual balance? The user can ask for this in many ways, “What is my current balance?”; “How much money do I have?”; “How much is in my checking?” All of these prompts need to return the same information, but proper language processing is required to understand that all of these prompts mean the same thing. It would be best if you used keyword matching alongside natural language processing. These high-variance prompts also require technology and engineering effort in the form of sentiment analysis.

Technical Capabilities

Experience Type

It goes without saying, but the type of experience you are creating will significantly influence the best technology to use.

For chatbots, you’ll need to consider the web or app experience in which the chatbot lives. What technologies are you using for the current web or app? Certain chat technologies work better with different web and app technologies and may narrow down your choices to a few options.
For text services, consider how you want to store and access chat logs, reports, and analytics. How broad is the audience? Is the chat intended for a select audience or set of users? Or is it more publicly targeted, available for anyone to use?
For voice, consider the device support you are targeting. Given the state of voice assistants, your options may already be narrowed to leading technologies from Amazon, Google, and IBM. That said, there is potential for other, lesser-known, or newer technologies available that may fit your needs.

Technical Skills & Needs

What level of development effort can you support? Will you have a dedicated team with the expertise to build the experience? If not, what is the team makeup, and how much time, effort, and resources do you have available to create the experience? If your experience requires even basic NLP/NLU or AI/machine learning, you will need both the skills and tools to support a more technical approach.

Data Management

What level of data management do you need both short and long term? There are multiple ways in which you can and should leverage conversational data, including:

Reporting: Both on the overall chat performance as well as general usage analytics
3rd-Party Use: Are you planning to intake/output 3rd-party data such as CRM and profile information? Will you be trying to leverage chat data for other technologies such as personalization engines or automated marketing tools?

Storage & Conversation Logs

Are you looking to store and access chat related data within another technology like a CRM or CDP? What information are you looking to keep regarding chat logs? How does security play into where your user information is stored and accessed?

This may seem like a lot of considerations, as chat experiences can be very simple or complex. Getting the right tool in place is essential for both the build and ongoing maintenance of your experience. You don’t want to select a tool that is more robust than your needs require. But you also don’t want to choose something that is not robust enough or cannot scale appropriately if and when the time comes.

When comparing conversational technologies, I like to use a comparison framework to compare the needed features and functionality. Below are some examples of technology research I recently conducted for a chatbot project:

Like most digital research, you’ll want to get in the weeds in the right areas.

A simple ranking or comparison system for needed features and functionality can help decision making by informing which technologies best serve short and long term needs.

Designing Your Experience

The complexity of your experience will dictate the effort required to design comprehensive conversational flows. Still, ultimately you are creating the same thing – all of the potential conversational flows and chat scenarios:

In this example, you see every possible interaction of a text-based chatbot – user inputs/input types, chatbot responses, content and copy formatting, fallbacks, and error handling.

Before you dive into mapping your conversations, recommended pre-work includes:

Defining Personality

Whether in voice or chat, your conversational experience needs a personality. Personality goes beyond just extending a brand voice. The breadth and variety of a chat’s responses touch every user interaction, and it is essential to strike a balance between functional and personable. Although you do not want something too stoic or repetitive, personalities that are too cheery or playful can be annoying or slow down the flow of conversation. Striking that balance can be difficult, so make sure to do your due diligence in defining the personality from front to back. Alongside that, you might have to create some personification or avatar for your experience. Should it be some kind of robot? A human avatar? Perhaps it’s some vaguely anthropomorphized version of your company’s logo? Though difficult to get just right, there are plenty of helpful resources out there to help define your chatbot or voice assistant’s personality:

Chatbots

How to Craft a Chatbot Personality (Without Damaging its Performance)

The Ultimate Guide to Chatbot Personality

Voice

Your Brand Has a Voice Personality Model Whether You Like it or Not. Here’s How to Define It

Does Your Digital Assistant Meet User Expectations for Personality?

Identifying Response Types

No matter your solution, you will need the following response types within your experience. Beyond the standard response types, you should identify other response types you will serve users:

Standard Response Types: Greeting

How the chatbot or voice assistant greets the user

General Responses: The standard responses to a user prompts, e.g., “Ok, here’s what I found for __________.”

Fallback Responses: The response when a user prompt is not understood, e.g., “I’m sorry, I’m not sure I understand what you are asking. Can you repeat or reword your question?”

Closing Responses: The displayed response when returning content/information. This response should include a follow-up mechanism to confirm the returned response is accurate and meets user expectations. (Example).

Feedback Mechanism: To better understand the performance of your experience, a feedback mechanism should be put in place at the close of a conversation. In chat and text, a simple positivity response works well. In voice, feedback mechanisms are used less often, at least through voice responses. If you’re using an existing device or platform, make sure to leverage any user-feedback features from reputable 3rd parties or that are natively available.

Other Response Types: *Please note that these responses typically apply to chatbots or text experiences.

Rich Messaging

In general, rich messages are visual and interactive content messages outside of simple text. This often includes card treatments, imagery, video, buttons, or other ways to interact with a bot’s content beyond the strict messaging.

In this LiveChat example, we see rich messaging in the form of interactive content cards showing images, descriptions, and CTAs to continue the chat.

Follow-Up Prompts

When a user prompt returns multiple results that are high-confidence matches, you will need to ask the user a follow-up response to better narrow down results. For this, you can use a basic multi-choice message or something more interactive, like responding with a word or phrase that best fits their needs.

In this Progressive Insurance chatbot, we see a follow up being used at the start of the chat because “get start” can mean many things based on “Flow’s” capabilities.

Unprompted Message

This isn’t necessarily a response type, but for bots that are persistent in a website or app, typically with registered users, unprompted messages can appear. For example, help tips and tricks from an application bot when it recognizes multiple errors. Or a promotional offer to loyal customers on an e-commerce site or app. (Example)

Idle State

When your chatbot or voice assistant retrieves information, think of the idle state the user sees. In messaging apps, the three-ellipses message is a typical pattern. For voice assistants, think of the hardware you are using and any visual control you can leverage. Smart speakers can include lighting-feedback or in the case of Siri and Google Assistant, built-in UI animations.

Miscellaneous

Are you planning a response type that doesn’t fit into the above types? Make sure you document and plan for these types of responses! Particularly what data, content, or information is required to return the response accurately.

Once the above considerations are complete, you’re ready to diagram your chat flows. This is relatively straightforward but can require a lot of effort to make sure you’ve covered all your conversational experience scenarios.

This example shows how each user input has both a response and a fallback, ensuring there are no dead ends.

To create a cohesive and natural flow of conversation, some considerations to keep in mind include:

No dead ends: No matter where a user is in the conversation, they should never be left feeling like they do not have a resolution. Even when an exchange does not yield the results a user is looking for, you should ensure that your experience offers up ways to continue looking for the results they need. Here are some references for chat fallbacks and voice design best practices to help your design efforts.

Keep it concise: Say the most with the least, both in copy length and in total messages within a flow. When done right, the benefit of conversational experiences should make tasks simpler. If it seems like users could accomplish a task more quickly alone, however, then you either need to rework your conversational flows or revisit whether or not chat is the correct way to handle that particular task or issue.

Patterns, not repetition: Every user prompt does not require a wholly unique response, but small variations in dialogue can make your experience feel more dynamic. Even changing a few words or swapping out a short phrase can go a long way. Though you should aim to provide variations in dialogue, you should stick to strict patterns when it comes to the overall conversational flow. Every user should be able to anticipate the type of response they will receive. Doing so requires establishing a consistent pattern for how user prompts are treated. For example, a rich message response can vary in copy and content but should be the same in how it’s returned to the user: Fetching results > return dialogue > return content > return follow up/confirmation.

- Use visual tools when possible: This makes it easier to design your flows and communicate how dialogue, content, data, and technology connect at all the conversation touchpoints. Many chatbot and voice tools have chat-design features and functionality baked into their products. If you are designing your conversations before picking a technology, popular tools include Miro, Mural, creately, draw.io, and many other modern flowchart tools.

Focus on one use case at a time: Designing your conversational flows can seem a bit open-ended and can be approached in various ways depending on your experience. Typically it is best to take a more linear approach and start with your most complex and potentially varying use case. If you’re designing the chat flows correctly, you will be hitting all of the building blocks of a chat session, which you can then leverage or evolve as needed. With a systematic approach, the design decisions around each use case can influence and leverage system-wide patterns to greater efficiency.

Keep in mind data and technology: For every interaction in your conversational flows, both from users and the chatbot or voice assistant, ensure that you have given thought to the data and technology needs to analyze and return every response.

Try to “break” your conversations: For every potential user input your experience might receive, the best way to ensure detailed and thoughtful execution is to consider every possible edge case you can encounter. This helps to understand the technology required to return the type of responses you want and design around complex or uncertain situations.

Collaborate early and often: Your team will likely have some level of involvement across multiple disciplines, from design and development to content, marketing, or branding. The last thing you want is to bring in someone too late and have to scrap or rework approaches because your designs are not feasible or meet various needs and goals. As you design, make sure every interaction has been reviewed and signed off across all respective disciplines related to their aspects of the overall experience.

Building Your Experience

As with all tech development, planning, communication, and information sharing is critical to a successful build. Unfortunately, there are too many unknowns and variables to give concrete and actionable guidance on building conversational experiences. In general, you should refer to the resources and documentation from the tools and technologies you will be using. Although I can’t provide detailed technical guidance, here are some useful tips and activities for you and your teams to go over before development gets underway:

Collect and Share Information: Make sure you have all up-to-date artifacts and information collected and stored in a single and accessible location.

Review and Get Sign Off: Everyone involved in creating your experience should have in-depth familiarity with the overall strategy, design, and technology decisions that are being implemented. Commitment by all team members is required to make sure significant disruptions are avoided throughout the build. Rather than relying on team members and stakeholders to review everything on their own, a workshop presentation is recommended to kick off the build phase. During the workshop, every discipline should speak to the goals, responsibilities, and outcomes of their respective work, and make sure everyone understands all critical information.

Create Smaller Milestones: Success for a conversational experience looks very different from standard web or software development. A lot of complicated and time-consuming work is required for functionality that is typically more straightforward in digital products. Create goals and objectives that reflect the complex nature of conversational experiences rather than aiming for the first testable-build. For example, your first milestones will likely revolve around data setup – configuring and testing webhooks, API calls, data synchronization, etc.

Detail Your Data Needs: This can’t be stressed enough, but the data considerations are foundational. Ensure all data needs are identified and scrutinized before working it into the build:

Data For The Experience: This includes native and 3rd party data related to creating the experience.

Data For Analytics and Report: This includes native and 3rd party data related to the conversational experiences performance and usage. Make sure you have a detailed plan for collecting the right data inside the experience and a method for then storing, formatting, and exporting/leveraging the collected data for your needs. Ahead of your build, make sure you’ve identified the goals, objectives, and KPIs you wish to track related to your chatbot’s performance, its adoption and usage, and any qualitative feedback and information you want to gather. If you’re unsure of what you types of data is useful to track, here are some resources to get started:

Establish Your Testing Plan: Testing chat and voice experiences can be quite tricky, as end-user behavior is nearly impossible to predict. If you’re using NLP, ML, or AI in your experience, it becomes more difficult. Due to testing complexity, it is crucial that test planning happens well before the build is complete, and modified as needed before testing occurs. While there are no established, best-practice methodologies or frameworks for testing conversational experiences, I’ve found that the work of others’ experiences is the best starting point::

What’s Next

Once your experience is built and tested, it’s time to get ready to launch! In part 3 of the playbook, we’ll go over the frameworks and considerations for launching your experience and how to monitor, maintain, and improve or scale your experience over time.

Tags: an event apart, smashing, speckyboy, UX Interview, ux stack, UX Stack Exchange, UXBooth

The Conversational Experience Playbook: Part 2 – Design And Build

Chat Experience