Not Just Anybody Should Do User Research

Ellen asked us if she could respond to Jonathan’s article from May: User Research May Be the Most Important Role in Your Company. We always welcome differing views and healthy debate at UXBooth.com.

As hard as it may be for some to accept, it’s time for the UX industry and the experience research community to come to some form of agreement. While experience research is finally fully recognized as critical to UX, there are too many UX novices and veterans who believe that “anyone” can conduct user research. Before the readers miss my point, gaining empathy for the customer must be part of the process and everyone on the team should participate in some way. Customer feedback is essential – but in the hands of non-researchers, the buyer must beware.

Dispelling User Research Myths

The UR industry is at a crossroads today; and while more and more organizations say they realize the need for the role, there are four misconceptions that some executives, hiring managers, and even UX professionals have about user research:

Myth #1: User research takes too long and costs a lot of money.

User research can be as simple or as complex as the project team wants it to be. Determining project time and investment before a study scope is understood indicates a lack of willingness, not a lack of resources. And most importantly, by understanding and empathizing with users from the beginning, the majority of projects are often simplified as unnecessary or unwanted features are abandoned.

Reducing and removing features based on user research may be difficult to sell to some product owners who are often rewarded based on the volume of features delivered, not on successful outcomes for users. But it’s worth the effort for an organization’s bottom-line. Researchers are often required to play catch-up after the fact and development teams then redesign unusable experiences, a far greater waste of time and money. The politics of usability is often what leads to this outcome; not the reality of conducting user research.

Myth #2: Anyone can do user research well.

As third-party user research platforms have exploded in the past few years and have expanded the universe for unmoderated research, the marketing message for these platforms is that anyone can do this. But research encompasses much more than just setting up a few tests and launching them: interviewing, analyzing, and synthesizing requires expertise. Asking the right questions up front is one small part of the equation as well as the psychology behind how the brain functions; figuring out why people take actions and make decisions should never be taken lightly.

What people say and what people do are very different things, to paraphrase Margaret Mead, and collecting the data is only the start to gaining actionable and valuable insights. Anyone can collect data – it’s knowing how to collect it, what to do with that data, and synthesizing the results into valuable actions that’s the hard part.

Myth #3: Any user research is better than none at all.

Not having a solid research plan, not understanding the problem to be solved, or not talking with the appropriate audience has led many a design and development team down the wrong path. Biased data, relying only on self-reported behavior and one-sided perspectives, are a waste of time and effort – and make it harder for experience researchers to surface and evangelize what is accurate and fact-based about user needs, expectations, and behaviors.

Myth #4: Designers or developers can do user research part time.

Human beings easily attach to the things they create, and expecting designers and developers to remain objective as they watch or hear criticisms of their creations simply doesn’t hold water in the real world. Experience researchers bring a level of objectivity and neutrality that is borne from not being directly attached to or invested in what is being studied. If a hybrid researcher-designer or researcher-developer is the only option for an organization, there must be clear lines of demarcation so that the research is not compromised.

Photo by You X Ventures via Unsplash

User Research Is Essential, But Where?

Healthy debate is always good but the UX research community needs to provide the vision so that we can move forward. Based on the growing interest of the past few years, some may think the field is in its infancy. But experience research has been around for decades, even though it often remains an afterthought or an act of desperation at many organizations. We’ve all been through it: there was no time or money for user research during the planning and development phases, but after launch, when no one is able to use the new product, suddenly there is time and money and a desperate need to know “why” the next best thing wasn’t used or usable.

While user research is viewed as essential–few can get away with saying it isn’t nowadays– the concept that anyone could or should conduct research continues to spread. Though well-intentioned, many on the project team do not have the training needed to conduct and facilitate unbiased, actionable research, regardless of their desire or interest in doing so.

The UX profession itself is sending mixed messages that are picked up and retained in the minds of cost-conscious hiring managers, senior executives, and product directors, some of whom do not understand the difference between UI and UX – let alone what we mean by empathy.

Often, these same individuals typecast UX researchers as roadblocks to the rest of the team interacting with customers. And UX researchers have a hand in this as well. As we’ve made our way forward, we have not done a good enough job of including non-researchers in the process and often approach research as academic or strictly a qualitative or quantitative endeavor, which has not helped the cause.

As UX research has gained acceptance, however, it does not seem to have stopped what Steve Krug called religious debates in Don’t Make Me Think.

“As a result, most design discussions are full of strong personal opinions, usually disguised as facts…it’s very appealing to have someone you can turn to for definitive answers. The odd thing is,” Krug said, “I wrote a book that spends most of its time explaining that there aren’t many definitive answers, just a few useful guiding principles.”

Religious debates are still alive and well in various conference rooms and board rooms around the world in 2019, as the scenario Jonathan Deesing described in his May article User Research May Be the Most Important Role in Your Company points out. Deesing’s tale — his team spun its wheels debating button text, everyone weighed in with an opinion until someone suggested bringing in user research — is a familiar scene to most anyone who has been involved in the design or development of a user experience, whether physical or digital.

The reality is that the user research role may be the most important role that does not yet exist or is not yet filled in your organization. Doing user research well requires a combination of skills, training, practice, and patience; and finding someone with experience is the solution, however difficult that may be. And to say that any research is better than none does a disservice to the field and underemphasizes the potential value user research can provide.

“If you follow some good principles you could get some decent feedback that might be helpful in a limited way; but it can’t be any feedback from just anybody. You need to constantly check biases and assumptions. More broadly, it’s about properly positioning the inquiry,” said Janine Coover, founder of J9Arts Research and a full time UX researcher with more than 10 years of experience working for dozens of clients. “You conduct research not to learn what you already know but to be open to uncovering something you hadn’t thought of. I approach every session with the mindset that I’m going to learn something new.”

Photo by Helloquence via Unsplash

It’s Time We Treat User Research as a Discipline

The assertion that you don’t need to be a user researcher to do user research on its surface seems straightforward and innocuous. But if we took that logic another step and said “you don’t have to be a coder to write code” or “you don’t have to know what a pixel is to design,” no one would take it seriously. And that doesn’t make user researchers bogeyman, as Deesing’s article describes them. It makes them experts and experienced in ways that non-researchers simply are not.

Krug was right when he said that usability studies are not rocket surgery. Most of the issues user researchers deal with come down to common sense or clashes between decision-makers, as Deesing pointed out. Organizations should recognize the fundamental human inclination to want to validate your own creation and then ignore anything that invalidates it, or what I call the politics of usability.

Over the years, as UX religious debates go on and on, the “let’s ask UR” last ditch effort has been used as a way to come to a decision, rather than the team making a decision based on what is known about customers before putting pixels to code. The data is everywhere – most teams have so much of it they don’t know what to do with it all. User research requires a hypothesis to validate (or nullify) and should not be used solely as a fishing expedition. After months of meetings that did not include a researcher, it then is a crutch for marketers, designers, and developers alike who either don’t know or can’t accept who their actual users are and what they need.

As part of a design-thinking mindset, the trinity of user needs, business assumptions, and technical feasibility have to be looked at together and should not be viewed in isolation. In order to reframe this conversation, business partners should be asked to provide assumptions rather than requirements. Some of those assumptions need user input, and some of them don’t.

Customer feedback is essential and everyone on the team should participate at some level in order to empathize with the human beings who will be using what we build. But let’s put the responsibility of conducting user research into the hands of skilled researchers. Keep holding conversations, sending out surveys, and connecting with customers – but let’s start calling that activity something else. Characterizing all customer feedback as “user research” needs to end in order for experience research to be treated as a discipline that is valued and incorporated in any development project.