# Talking to Users: The Mistake That Killed Our Roadmap > Customer interviews are supposed to ground the roadmap. Done wrong they hijack it. The trap I keep watching teams fall into, and the discipline that stops it. Author: Abdul Hannan Shaikh, senior technical PM at APIMatic. Published: 20 November 2024 Topic: PM craft Reading time: 4 min read Canonical: https://soch.ahshaikh.com/blog/talking-to-users-mistake-that-killed-our-roadmap/ --- A team I worked with ran 30 customer interviews ahead of a major roadmap planning cycle. The interviews were good: open-ended, recorded, transcribed, tagged. The team felt rigorous. Three months later, the roadmap they built from those interviews shipped, and the metrics it was supposed to move barely budged. The interviews weren't the problem. The way they got translated into roadmap items was. Here's the trap. ## The trap: feature requests as truth Ask a user "what would make this product better for you?" and what comes back is a feature request. Features are easy to articulate. They're concrete, they have a clear noun, they fit in a tweet. Pain is harder. Pain sounds like "I dread doing this on Mondays" or "I have an Excel macro that nobody at the company understands." Build the roadmap from feature requests and you're optimizing for whatever users happened to be able to describe. The interviews drove the roadmap. They just drove it at the surface of the problem. The team I described above had a roadmap full of items every user had specifically asked for. Each item shipped. Each was used by the user who requested it, briefly. None of them moved retention, expansion, or NPS, because none of them addressed the actual reason users were churning. ## The discipline: separate evidence from interpretation Fewer interviews wouldn't have helped. The gap between what a user said and what it means is where the discipline has to go. I split interview notes into two columns. The left is verbatim quotes, the exact words. The right is my interpretation: what I think the user is actually struggling with, why it matters to them, what the underlying need is. The columns never get blurred. The verbatim is evidence; my interpretation is hypothesis. When the roadmap conversation happens, both columns matter. Verbatim quotes give a stakeholder a visceral sense of what users are saying, and they're more persuasive than any chart I've ever built. The interpretation column is the one that gets debated. Two PMs can look at the same quote and disagree about what it means, and that's the conversation worth having. ## The five-whys checkpoint Before any feature request becomes a roadmap item, I run it through one filter: can I trace it back to a job-to-be-done or a measurable user outcome? If the chain is "user asked for X, ship X," that's a feature factory chain. If the chain is "user struggles with Y because Z, which makes them spend 2 hours/week on what should be 10 minutes, and feature X reduces that to 15 minutes," that's a roadmap item with a metric attached. Most feature requests fail this filter. The work the request implies might still be worth doing, usually as a piece of a larger initiative. And often the right move is to build something no user asked for, because the underlying job turned out to be a different job. ## What users are unreliable about Even when the interviews are well run, there are things users are systematically unreliable about. Learn to discount these and the signal gets better. **What they will pay for.** Hypothetical pricing is noise. People say they'd pay for things they wouldn't, and it happens just as often in reverse. Real pricing signal comes from real purchase decisions, or from willingness-to-pay studies with skin in the game. **How they would behave with a new feature.** "I would use that every day" rarely survives contact with reality. Watch what they already do instead: the current substitute behavior, the hacks they've built, the things they Excel-around. That predicts adoption. Aspiration doesn't. **What they don't notice.** This is the expensive one, and it's the hardest to design an interview around. Most users can't articulate friction they've already absorbed into their workflow. The third manual step in onboarding feels normal to them by now, so they will never list it as a problem. You have to watch the workflow happen, on a recording or in a co-pilot session, before the thing nobody mentioned shows up. ## The takeaway Customer interviews are a forcing function for empathy. The roadmap source-of-truth is your interpretation of patterns across many interviews, weighted by what you can measure and what you can ship. The team I started with rebuilt its roadmap six months later. The second time, they ran fewer interviews, separated evidence from interpretation, and tested three of their interpretations with prototype probes before committing to roadmap items. The next quarter, the metrics moved.