A Guide to User Research Analysis

When designers perform user interviews, field observations, or usability tests, they gather tons of notes and data to help inform design decisions and recommendations. But how do they make sense of so much qualitative data? Talking to customers is great, but most people walk away feeling overwhelmed by the sense of more information than they know what to do with. Learning how to properly analyze UX research helps turn raw data into insights and action.

 

What Is User Research Analysis?

User research analysis is a vital part of any research process because it is the very act of making sense of what was learned so that informed recommendations can be made on behalf of customers or users.

As researchers conduct analysis, they’re spending time categorizing, classifying, and organizing the data they’ve gathered to directly inform what they’ll share as outcomes of the research and the key findings.

 

Why Should Researchers Spend Time on Analysis?

Our natural instinct is to believe we can remember everything we heard or saw in an interview. But following impulsive decisions made from raw notes and data can be misleading and dangerous. Recommendations based on a single data point can lead a team down the path of solving the wrong problem.

Further, doing so is simply reacting to data, not making sense of it. This can cause companies to focus on incremental improvements only and miss important opportunities to serve customers in more meaningful, innovative ways.

A great example of this is when we see teams sharing research findings like, “6 out of 10 people had difficulty signing in to our application.” On the surface, a reasonable recommendation could be to redesign the sign-in form. However, proper research analysis and finding the meaning behind what that data represents is when the real magic happens. Perhaps the reason people had trouble signing in was due to forgotten passwords. In this case, redesigning the sign in form wouldn’t necessarily solve this problem.

Performing the necessary analysis of user research data is an act of asking “why” the “6 out of 10 people had difficulty signing into the application.” Analysis transforms the research from raw data into insights and meaning.

Consider what Slack did with their sign-in process. Slack allows a user to sign in by manually typing their password or having a “magic link” sent to their email which the person simply needs to click from their inbox.  They get signed in to their Slack team and get started.

This decision wasn’t an accident; it came from a deep understanding of a customer pain point. That deep understanding came from making sense of user research data and not simply jumping to a conclusion. Slack’s example demonstrates the power of spending time in analyzing user research data to go beyond reacting to a single observation and instead understanding why those observations occurred.

 

When to Do Research Analysis

Before the Research Begins

Great analysis starts before research even begins. This happens by creating well-defined goals for the project, research, and product. Creating clear goals allows researchers to collect data in predefined themes to answer questions about how to meet those goals. This also allows them to create a set of tags (sometimes known as “codes”) to assign to notes and data as they conduct their research, speeding up analysis dramatically.

Before any research session begins, craft clear goals and questions that need to be answered by the research. Then brainstorm a list of tags or descriptors for each goal that will help identify notes and data that align to the goals of the research.

 

During the Research

Researchers often tag or code data they gather in real time. This can be done multiple ways using spreadsheets, document highlighting or even a specialized research tool like Aurelius.

When taking notes in a spreadsheet, tags can be added to individual notes in an adjacent column and later turned into a “rainbow spreadsheet.”

For teams physically located in the same space, an affinity diagram with sticky notes on the wall works well. Here, each note can be added to an individual sticky note with top level tags or themes grouped physically together.

 

 

There are also software tools like Aurelius that help researchers tag and organize notes as they’re taken which also makes for quick viewing and analysis of those tags later.

 

It’s also useful for teams to have a short debrief after each research participant or session to discuss what they learned. This keeps knowledge fresh, allows the team members to summarize what they’ve learned up to that point, and often exposes new themes or tags to use in collecting data from the remaining research sessions.

 

When the Research Is Done

This is where most of the analysis happens. At this point researchers are reviewing all the notes they’ve taken to really figure out what patterns and insights exist. Most researchers will have a good idea of which tags, groups, and themes to focus on, especially if they’ve done a debrief after each session. It then becomes a matter of determining why those patterns and themes exist in order to create new knowledge and insight about their customers.

 

How to Analyze User Research

Tag Notes and Data as You Collect It

Tagging notes and data as they’re collected is a process of connecting those tags to research questions and the research questions back to the project or research goals. This way you can be confident in the tags and themes being created in real time. Here’s how to make the connection between tags, research questions and project goals.

Imagine the research goals for the project are:

1. Increase the number of people signing up for our product free trial
2. Increase the number of people going from free trial to a paid account
3. Educate trial customers about the value of our product prior to signing up for a paid account

From there, research questions can be formed such as:

• “Does the website communicate the right message to share the value of a free trial?”
• “Is it easy for a new customer to sign up?”
• “Are new customers easily able to start a free trial and begin using the product?”

From those questions, we can extract topics and themes. Since we’re researching the free trial, sign up process and general usability of that process, they become clear choices for tags. Also, since the research is meant to answer a question about whether or not potential customers understand the value of our product and free trial, this too provides a clear topic and tag we can use. So, useful tags based on those questions would be:

• #free-trial
• #value-prop
• #signup-reason
• #signup-process
• #onboarding
• #confusion

As the team conducts the research, they can tag notes and observations according to those themes that align to the high level goals and questions for the project. All of this highly increases the ease and effectiveness of research analysis later.

 

Analysis After Each Session

A common user research practice is for the team to debrief after each interview, usability test, or field study to discuss what was learned or observed. Doing this while also reviewing the notes and observations helps researchers hear the same information from a new perspective.

Let’s imagine the team found the following patterns while conducting their research:

• Potential customers visited the product page, free trial sign-up page, and went back to the product page several times prior to starting a free trial.
• Some people had multiple browsers open with competitor sites pulled up while signing up for the free trial.
• Potential customers mentioned waiting for the “right time” to start their free trial on several occasions.

This may help the researchers create new tags (or codes) for remaining sessions, such as:

• #right-time
• #competitor-review
• #feature-comparison

Using these new tags adds another dimension to analysis and provides deeper meaning to patterns the team is finding. You can see how the combination of these tags and themes already begin to paint a picture of customer needs without any detailed notes!

Here are some good tips for knowing when to tag or code a note:

• It aligns directly to a project/research goal.
• The participant specifically said or implied that something is very important.
• Repetition – a thing is said or heard multiple times.
• Patterns – when certain observations are related or important to other tags and themes already established in the project goals or research.

 

Steps for Analyzing Research Once It’s Done

Once all the research is done, it’s time to dig in to find patterns and frequency across all the data gathered.

Step 1 – Review the notes, transcripts, and data for any relevant phrases, statements, and concepts that align to the research goals and questions.

Step 2 – Tag and code any remaining data that represents key activities, actions, concepts, statements, ideas and needs or desires from the customers who participated in the research.

Step 3 – Review those tags and codes to find relationships between them. A useful tip for this is to pay close attention to tags that have notes with multiple other tags. This often indicates a relationship between themes. Create new tags and groups where appropriate to review more specific subsets of the data. Continue this process until meaningful themes are exposed. Once that happens, ask questions like:

• Why do these patterns or themes exist?
• Why did participants say this so many times?
• Does the data help answer the research questions?
• Does the data inform ways to meet the research goals?
• Does one tag group or theme relate to another? How? Why?

 

Sharing Key Insights from User Research

A key insight should answer one or more of your research questions and directly inform how to meet one or more of the established business goals. When sharing key insights, be sure to make a clear connection between one of the business goals, research questions and why the key insight is relevant to both. The most effective way of turning research into action is by helping teams make a connection between key insights and business outcomes.

3 Parts to a Key Insight

There are three parts to creating a key insight from user research:

1. Statement of what you learned
2. Tags that describe the insight (often used from the analysis, but can also be new tags entirely)
3. Supporting notes, data, and evidence that give further context to the key insight and support the statement of what was learned

A key insight from the example project might be:

“Prospective customers are worried they might not have enough time to review our product during the free trial.” #right-time #signup-process #free-trial

This represents the pattern observed of customers mentioning the “right time” to sign up for a free trial and comparing the product to competitors. It also goes beyond sharing the quantitative data that those things occurred and offers a qualitative explanation of why they happened. All of this leads to clearer recommendations and the ability for other teams to take action on the research findings.

Creating key insights from the research in this way allows for the most effective sharing and reuse later. By providing supporting notes to each insight, stakeholders and others consuming the research findings can learn more detail about each key insight if they so choose.

 

Next Steps for User Research Analysis

Conducting detailed analysis of user research data helps teams clearly share what was learned to provide more actionable recommendations in design and product development.

Here are some tips for making user research analysis faster and easier on upcoming projects:

1. Begin the user research by creating well defined questions and goals.
2. Create tags based on each goal.
3. Tag research notes and data as it’s collected to speed up analysis later.
4. Debrief after each research session.
5. Review the data once research is finished to find patterns, frequency, and themes.
6. Make statements about each pattern or theme that was uncovered, describing what it means and why it’s important (aka: create key insights).
7. Share the key insights!