Lesson: Finding Customers with Cindy Alvarez

Step #8 Post-Interview: Remember your hypothesis

What do we do with the pile of notes? Yeah, that's the big other question. We tend to do team debriefs where we'll kind of go through the notes and call out the things that were particularly resonant and people were upset about this, people really like this. Did you notice that a lot of folks said that they have this problem? And just kind of saying, setting it aside as a known insight, like we have this insight, we have this insight. And when we have note takers, actually we'll tend to do a quick debrief with that person after the interview.

So we hang up the phone and then the researcher will sit with the note taker and say, "What really struck you about that? What really stood out about this?" And then just getting that kind of immediate impression and literally just writing that down like, "Wow, I was really angry. Wow, this person was really bored. Wow, this person doesn't understand technology at all." Whatever that thing is, just kind of highlighting that and then personally I like to go through my notes and kind of just circle the things that stood out the most. And right afterwards is when you really remember that.

But then you're basically just looking for evidence and some people literally will take sticky notes and like put on tally marks. Like this is how many people say that they don't have a budget. This is how many people have this problem. You can also do it in sort of inexact way but the main thing is just to kind of keep sharing that out with the rest of your team. It was like, "Hey, this week we learned these things." And people will ask questions a lot of times. "What do people say about this? Well, why didn't you ask this?" And that's just a really good conversation to keep having.

And a lot of times it's inexact and so it does become important to figure out how to tell a narrative around this and say like, "We keep hearing these things." And there's no magic number for the third person says that you do it. The fifth person says that you need to do it. Sometimes things will gradually build up overtime like every tenth person says this thing and then after a while you're like, "You know that's a lot of tenth people, we should start doing it." One of the biggest challenges is figuring out how to deal with the new information and it is sort of threatening. And one of the ways I found is to have everyone involved in the hypothesis generation and to actually write down, "Here is what we believe at this point in time."

And when we get information that is contrary to that to literally cross it out or write a new version underneath so that we can see on day one we all believed this. Now after day seven, we don't believe X anymore and here's the evidence to it. So you might say "Our original hypothesis was X, we all did customer development interviews, we now believe that this assumption was wrong and here's the evidence to say we talked to this person and they had said this. We asked this person how they buy a software and actually they don't have a PO process. They put things on their corporate card, or they had no budget. We thought that they had a budget but they actually don't." And to literally kind of point back to all those pieces of evidence. But also to keep reinforcing that the objectiveness of it like our hypothesis is this. Now our hypothesis is this.

Basically when you have a product or a feature or a design or anything that's subjective people start having ownership over it and it becomes very contentious to basically say, "The equivalent of your design is wrong. My design is better." That is not good for team dynamics and people tend to just dig in their heels. What we're not doing is saying like design A versus design B is better or product A versus product B is better. Our objective is to solve this problem. What's your building doesn't actually solve this problem. Or what you're building was a very good solution to this problem, but what we just discovered is that that's not the problem. So now here's our new objective problem and let's look at we are building. Does it solve the new objective problem?

And at that point people are going to say, "Well, no. Well, we could make it." And then you kind of say like, "Are we going to force it into something so that we get a not very good solution to this problem?" And make it clear to people that's actually what you're doing. Now that goal lines have changed, we have a new objective problem and if we keep building this thing we're already handicapping ourselves. We're already building a subpar solution because it was the solution to some other problem. And so always returning to that objective language can be very useful because then it's not about you guys built the wrong thing.