Skip to main content

User Research Questions

Questions & Answers to User Research questions. Pull requests for suggestions and corrections are welcome!

General advice: Know key-performance indicators (KPIs) and how to measure them. Know how to create and conduct research plans, research methods, and how to analyze data.

Table of contents​

What is UX Research?​

User experience research is the scientific process of collecting, analyzing, and synthesizing information to increase our understanding of users' behaviors, needs, and pain points to dictate the user experience. It's a crucial component of the human-centered design process and essential to creating solutions that meet user expectations and deliver value to customers.

References​

Why is UX research important?​

.. a vital outcome of research (not just the data that is gathered but the experiences we have in gathering it) is learning about what we didn't know that we didn't know. The most important insights often lie beyond the questions that we started with.

Brad Nunnan - UX Research, p.xi

Research is all about answering questions. It's about learning and understanding.

  • Introduces scientific rigor & process to design. It's the difference between designing for humans with observable truths and having an opinion.
  • UX Research provides data and insights to support, justify, and defend design designs to stakeholders.
  • Helps understand user behaviors and make better, more informed business and design decisions.
  • Helps decide which problems and features are the highest priority and how you might go about solving them.
  • Reduces the risk of building the wrong thing or building the right thing in the wrong way, saving time and money.
  • Removes designer bias by challenging assumptions.
  • Test and validate concepts & ideas early to avoid wasting time or adding features that users don't want or need.
References​

When do you conduct UX research?​

UX Research should be conducted at all stages of the design process; however, it's the most important at the beginning of any project or feature. The earlier the research, the more impact the findings will have on your product, and the more money and time you save further down the design process.

What kind of UX research methods and processes do you use when starting a new project?​

Generally, formative & generative research methods. These are field studies, user & stakeholder interviews, competitive analysis, etc. But, it depends.

The research methods must be appropriately selected to answer the research questions while taking into consideration the project timeline, budget, and the amount of synthesis required for that method versus another one. Research methods depend on the research goal, current stage in the design process, and the available resources.

Start by asking good questions. Create a research plan with clear research goals, and hypotheses. Select research methods. Plan logistics. Recruit your research team. Analyze findings and synthesize insights. Report your findings.

References​

What UX research methods are you familiar with? Briefly explain their purpose and when you would use them.*​

I'm familiar with a lot of them, but don't have experience with all of them.

There are a variety of different research methods used in UX. Instead of trying to remember all of the research methods, it's better to understand the types of research methods, the type of data they result in, and when these categories of methods should be used. For example, quantitative vs qualitative, generative vs evaluative, and formative vs summative research.

However, some of the most popular research methods involved in UX are:

  • Field Studies
  • User Interviews
  • Cognitive Walkthroughs
  • User Personas
  • User Journey
  • User Surveys
  • Empathy maps
  • Tree Testing
  • Card Sorting
  • User flows
  • Task Analysis
  • A/B Testing
References​

Explain the differences between idea validation, design validation, and design verification.​

Idea validation refers to determining whether your idea has any value. Idea validation usually occurs during the initial research phase to determine whether the idea should be further pursued or researched before devoting resources toward its design and development. Idea validation may involve market validation, competitive analysis, engineering feasibility, and determining if the product idea matches the company's goals, brand, vision, or culture.

The following terms are borrowed from the FDA Design Control Regulations to clarify some of the confusion around UX validation.

Design validation is about whether or not you designed the right product. Design validation ensures that the product conforms to the defined user needs and intended uses and shall include testing of production units under actual or simulated use conditions. Design validation is crucial during the design stage to support the design direction and in production to ensure the final product meets the actual user needs and requirements. Design validation usually involves approval from all stakeholders to begin development.

Design verification is about whether you developed the product right. Design verification confirms that the design output, the final product, meets the design input or design specification sheet. Design verification occurs during or after development to verify the product meets the design specifications. In UI design, this can mean checking whether the developed UI component matches specifications in the design files and running and passing all unit tests.

References​

How do you validate or conduct usability testing on a design?*​

In design validation, we need to demonstrate that the product meets the user’s needs.

Validation testing would include test cases, test suites, or even clinical trials designed to prove that the product, as built, operates according to the user’s expectations under the conditions where they intend to use it.

Since these tests should be run on production or production equivalent units, design validation tests are often the last tests performed. However, design validation during the design process can help catch potential design flaws and ensure the design is progressing in the right direction.

Some design validation test methods are:

  • Cognitive walkthrough
  • Heuristic Evaluation
  • Usability Lab Studies
  • Clickstream Analysis
  • Eyetracking
  • SUS
  • GOMS
References​

Explain p-value.*​

P-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct.

The smaller the p-value, the greater the evidence against the null hypothesis and the greater the statistical significance of the observed difference. A p-value of 0.05 or lower is generally considered statistically significant.

A p-value greater than 0.05 means that deviation from the null hypothesis is not statistically significant, and the null hypothesis is not rejected.

References​

What are qualitative and quantitative research?*​

Qualitative research is the study of anything subjective, notably the personal stories and challenges of our customers. Where quantitative research focuses on what can be measured, such as the time to complete a task, qualitative research looks at why customers are completing the task in the first place. Qualitative research seeks to understand customers’ motivations and desires by focusing on comprehension and accessibility that might not be numerically measured, but can nevertheless impact the usability and desirability of systems.

Quantitative research is simply defined as the study of what can be measured and observed. Quantitative research doesn’t tell us how to fix things, doesn’t tell us why things happen, and doesn’t share information that isn’t asked for.

References​
  • Nunnally, Brad, and David Farkas. UX Research: Practical Techniques for Designing Better Products. O'Reilly, 2017.

What’s the difference between qualitative and quantitative data? When should you use them?*​

Quantitative data are used when a researcher is trying to quantify a problem or address the "what" or "how many" aspects of a research question. It is data that can either be counted or compared on a numeric scale. This data is usually gathered using instruments, such as a questionnaire which includes a rating scale or a thermometer to collect weather data. Statistical analysis can be more easily performed on quantitative data.

Qualitative data describes qualities or characteristics. It is collected using questionnaires, interviews, or observation, and frequently appears in narrative form. Qualitative data may be difficult to precisely measure and analyze. The data may be in the form of descriptive words that can be examined for patterns or meaning, sometimes through the use of coding. Coding allows the researcher to categorize qualitative data to identify themes that correspond with the research questions and to perform quantitative analysis.

References​

What is generative research? Evaluative research?​

These methods are categorized by the intention with which research is conducted.

Generative research also known as exploratory research is used to discover people’s motivations, pain points, behaviors, and more. Generative research helps identify and define a problem and offers opportunities to create and explore new designs through research.

Evaluative research is used to evaluate and assess a product or concept, generally by collecting data such as KPIs or benchmarks that help improve your solution. There are two types of evaluative research: formative & summative.

References​

What is formative research? Summative research?​

Formative and summative research are types of evaluative research - meaning both are test methods used to evaluate and assess your product and design. The difference again here is intention.

Formative evaluations focus on determining which aspects of the design work well or not, and why. These evaluations occur throughout a redesign and provide information to incrementally improve the interface. Formative evaluations help formulate new or improve existing designs.

Summative evaluations describe how well a design performs, often compared to a benchmark such as a prior version of the design or a competitor. Summative evaluations summarize designs. Summative evaluations involve getting the big picture and assessing the overall experience of a finished product.

Both methods can be used in design validation.

References​

What is the difference between attitudinal and behavioral research?​

Attitudinal research is about what people say. Attitudinal Research refers to asking people about their opinions. These are self-reported data primarily used in market research as it collects information about people's stated self-beliefs. Things like surveys, focus groups, and preference tests are examples of Attitudinal Research.

Behavioral research is about what people do. It involves direct observation of the actions that a person takes. Ethnographic studies, usability studies, A/B tests, and eye tracking studies are all examples of behavioral research methods. In UX, behavioral research and data are more important, especially in usability studies.

References​

What are the difference between a field study, an ethnographic study, and a contextual inquiry?​

A field study is a general method for collecting data about users, user needs, and product requirements that involves observation and interviewing. Data are collected about task flows, inefficiencies, and the organizational and physical environments of users.

Investigators in field studies observe users as they work, taking notes on particular activities and often asking questions of the users. Observation may be either direct, where the investigator is actually present during the task, or indirect, where the task is viewed by some other means like a video recorder set up in an office. The method is useful early in product development to gather user requirements. It is also useful for studying currently executed tasks and processes.

Ethnography studies are a type of research method adapted from the field of anthropology in which anthropologists would study different cultures by immersing themselves in the culture for months or years at a time. Design research cannot accommodate the same kind of budgets and schedules as traditional ethnography; however, the principle of in-person observation of behavior remains at the core of ethnography as a research method. In true ethnography, a researcher will spend one or more entire days with a research participant, profiling the person’s life from waking up in the morning to going to bed at night.

A contextual inquiry is a semi-structured contextual interview that is conducted in the context in which the behavior of interest occurs. Because users are interviewed in their own environments, the analysis data is more realistic than laboratory data. This technique is generally used at the beginning of the design process and is good for getting rich information about work practices, the social, technical, and physical environments, and user tools. However, a contextual inquiry is still an interview and thus contains self-reported data, so it has the weaknesses of all self-reported (attitudinal) data.

References​

How do you perform research without users or if you don't have time for research?*​

  • You can use secondary data.
  • Use Heuristics
  • Do literature reviews
  • Outsource it
References​

What are some common UX research pitfalls & mistakes?​

  • Not performing research.
  • Not involving or testing on actual users.
  • Substituting market research for user research.
  • Over-representing users
  • Research is too broad and may lead to unactionable insights.
  • Not interpreting the data to correctly synthesize insights.
  • Not validating the research.
  • Research has bias.
  • Not having clear research goals.
  • Not selecting the right research methods.
  • Not using the research.
  • Starting research too late.
References​

How would you account for bias?*​

First, be aware of how bias can occur in research. There are a few entry points.

  • Cognitive bias in UX researchers can introduce biases during their interpretation of findings and behavior in user interviews.
  • In user interviews, asking leading questions may point users to the type of answer you're looking for. Ask more open-ended questions instead and be careful about influencing the interviewee through body language.
  • Standardize processes in user interviews and in analyzing data. Have different people come and analyze the data individually from their own perspectives. Increase sample size and randomly select participants.
  • Challenge all assumptions that are not supported by data. Have people from multiple backgrounds and diversity to make sure there is expertise in every aspect of UX.
  • If you feel strongly about a feature you are designing or have any other conflicts of interest, you may consider removing yourself from the research or be mindful and purposeful about framing questions from a neutral POV.
  • Additionally, you could select methods that have a lower risk of bias such as direct observational studies.
  • At the end of your research, conduct a debrief and come to an agreement or alignment about research findings.
  • As a project lead, try to be neutral and not judgemental. Have an open door policy, allowing your team to come to you and be able to talk freely without reservation.
References​

What are KPIs and how do you measure them? Why are they important?*​

KPIs are used to measure and evaluate the success of your product.

See this question.

[↑] Back to top