You might be doing user research wrong
An important truth about user research questions and a guide to getting better answers
On an impossibly sunny day in Los Angeles, I sat in a loft in Venice observing a group of twenty-somethings talk about their dating lives. The focus group moderator asked, “Next time you’re single, which app will you use to meet someone?” Not one person mentioned Tinder. How could that be? At the time, Tinder was the largest dating app—by a large margin.
Later in the conversation, the moderator asked, “Have you met anyone on Tinder in the last year?” Nearly everyone raised their hand.
This stark disconnect reveals a fundamental truth about user research: not all questions are created equal.
Some questions lead to the truth, and others lead to wildly inaccurate conclusions.
Product teams invest significant resources into user research with the hope of uncovering insights that will guide product decisions. Yet, too often, we fail to recognize that the nature of our questions dramatically affects the reliability of the answers we receive.
Let’s look at why.
The 3 user research questions
Product managers are like detectives trying to solve the mystery of user behavior. The quality of evidence we collect depends entirely on the questions we ask. I've come to realize that every user research question is really just asking one of three things:
What did you do?
What will you do?
What are you doing right now?
User research asks users to recall what they did in the past, imagine what they will do in the future, or narrate what they are doing in the moment.
“What did you do?”
These questions ask users to recall and describe their past behaviors and decisions:
“Have you met anyone on Tinder in the last year?”
"What product did you use to solve this problem before?"
"Walk me through the last time you booked a flight."
The quality of these answers tends to be high. People have a good memory of what they actually did in the past. They can recall those actions, and rationalize why they made those decisions.
In finance, past performance is not indicative of future results—but that’s not the case in product. Historical data paints an accurate picture of user behavior and user motivations. We can use that understanding to predict how users will behave in the future and what our product should do in light of those behaviors.
“What would you do?”
These questions ask users to imagine future behaviors, decisions, or benefits they might receive:
"Next time you’re single, which app will you use to meet someone?”
“Would you use this feature?”
"How much would you pay for this service?"
As product builders, we conduct user research because we want to predict the future—we want to be confident that what we build today will be useful to users tomorrow. So, naturally, we ask the user about that future.
In my experience, a very large number of user research questions fall into this category. The problem? Human imagination is flawed. There's often a wide gap between what we think we’d do and what we actually do.
This is particularly true when virtue signaling is involved:
Of course, I’ll invest significant money into my health—I’m a healthy person.
Of course, I’ll use the product my company spent so much money deploying.
Of course, I’ll buy local even if that means spending a little bit more.
Sometimes, the answers to “What would you do?” questions aren’t just inaccurate—they are the polar opposite of what users will actually do.
Still, these questions have their place. Human imagination has flaws, but also tremendous power.
I vividly remember a user research session when I was at Facebook. We were interviewing teens to understand what they wanted from social media. A wildly enthusiastic teen boy insisted that Vine videos (an early form of short-format vertical video) were the future—that was a year before anyone in the U.S. had heard of TikTok and a couple years before Instagram Reels.
His prediction was especially puzzling because Vine had shutdown about a year earlier. But he was undeterred, regularly hunting down Vine compilations on YouTube to share with friends.
Not every user research session foreshadows a multi-hundred billion dollar opportunity, but “What would you do?” questions can help you tap into your customers’ imaginations—just be careful to understand the limitations of that approach.
“What are you doing?”
These questions ask users to narrate their current actions and thoughts while using a product or prototype:
"Can you talk through what you're doing as you try to complete this task?"
"What are you thinking about as you look at this screen?"
"How are you deciding which option to select?"
When users think aloud during a product experience, they provide a direct window into their mental processes—capturing motivation, confusion, delight, and frustration—all in real-time. This creates a goldmine of insights that past and future-focused questions simply can't deliver.
A user saying "I'm confused about where to click next" or "I assumed this button would do something else" reveals the raw truth of their experience—not the sanitized version they'll report later or the idealized interaction they imagine having in the future.
The magic of "What are you doing?" questions lies in their ability to bypass the user's internal PR team. We all have one—that voice that rationalizes our past decisions and portrays our future selves as more logical, disciplined, and virtuous than we actually are.
Without time to activate this filter, genuine reactions emerge naturally, giving product teams authentic feedback that dramatically improves design decisions. Returning to our dating app example: watching someone swipe through profiles—on their actual account—reveals far more about their preferences and behaviors than any retrospective or hypothetical question ever could.
Predicting the future, one question at a time
The most effective product managers aren't satisfied with running standard focus groups and ticking research boxes. They're relentlessly curious about user behavior, leveraging every possible opportunity to gather authentic insights that drive meaningful outcomes.
To transform your user research from merely informative to genuinely predictive, consider these strategies:
Classify your questions by whether they ask about past, present, or future behavior—understanding this distinction is your first step toward better research design:
Recall questions ask “What did you do?”
Imagination questions ask “What would you do?”
Narration questions ask “What are you doing?”
Prioritize Recall and Narration questions. What users have actually done (past) and what they're doing right now (present) will consistently provide more reliable indicators than what they imagine they might do.
Use Imagination questions strategically, not as your primary research approach. When you do ask these questions, dig deeper into the "why" behind the answers—the motivations often reveal more truth than the stated preferences.
Scrutinize findings by tracing them back to the question. Questions about actual behavior, past or present, can be taken at face value. Speculative questions need more interpretation.
Important: Sometimes Narration questions can inadvertently become Imagination questions—especially when participants are asked to use products or complete tasks outside their typical behavior. Consider this example: would you conduct user research with men for a menstrual cycle tracking app like Flo? Of course not. Yet teams often make similar mistakes—testing with participants far removed from their ideal customer profile or asking people to imagine unrealistic scenarios, like planning trips well beyond their budget or comfort zone (a challenge we had to watch out for at Tripadvisor).
Great questions → product breakthroughs
Remember those twenty-somethings in Venice who swore they'd never use Tinder again? The same ones who minutes later admitted they'd all met someone on the app in the past year? That disconnect isn't just an amusing story—it reveals everything about how we should approach user research.
When we understand the difference between past, present, and future questions, our research transforms from guesswork to insight. The pattern is clear: what users actually do consistently trumps what they claim they'll do. Historical behavior provides a solid foundation for product decisions, while imagined future behavior is often quicksand.
Take a moment to review your last research protocol. How many questions asked about what users have done? How many explored what they were doing in the moment? And how many tried to predict what they might do? I bet you'll discover you've been leaning heavily on "What would you do?" questions—the least reliable type—without even realizing it.
Great products are built by observing & predicting real behavior, not on aspirations or false promises. That gap between what people say and what they do isn't just a research curiosity—it's exactly where your most valuable product opportunities are hiding.
What about you? Which user research questions have revealed the most surprising or valuable insights for your products? Share your experiences in the comments below.