User research is one of the most important and yet most overlooked areas of product work in early tech startups today. Based on numerous conversations I’ve had with startups of a variety of sizes, the majority report doing very little user research, some none at all. The good news is that most admit knowing they should do more, and they want to do more. So why aren’t they?

I believe the three biggest factors are:

  • Blind conviction. Simple cognitive biases prevent us from validating our assumptions and challenging our thinking.
  • Resource constraints. We are so focused on shipping that it often seems easier to just build, build, build, rather than take time to make sure we are building the right thing.
  • Lack of ownership and process. The resource problem is compounded by a lack of clarity in organizations around who owns the user experience and when to include user research.

In this post I’ll describe how to introduce simple user research methods at three different stages of product development.

  • Stage 1: Strategic product direction What goals are you helping users achieve? What is your value proposition? What problem are you solving? Who is your target user?
  • Stage 2: Product development What is the right product to deliver on your value proposition? How does it work? What set of tasks does a user need to do? What features does it need?
  • Stage 3: Optimization How can you make the product easier and faster to use? How can you improve metrics such as sign-up completion, time spent in the app, or friend referrals?

##Stage 1: Strategic Product Direction

###Validate your convictions

Startups are founded on convictions. Convictions from founders about how they see the world and the future. Convictions are great, they get us through the trough of sorrow and into the promised land. However, strong convictions and beliefs can also cause blind spots, making us slow to react to information that isn’t aligned with our narrative.

The solution? Avoid being reactive in the first place. Instead, validate and refine your convictions by continuously gathering facts and insights, and bake these methods into your product process. Before you ever start building your product – and then every fews months going forward – you can conduct simple user studies that will help you steer your product in the right direction.

###Answer big picture questions with qualitative user research

A simple approach to get started is to assess your current knowledge of your product space by bucketing it into three categories:

  1. Things you know to be true
  2. Things you think are true
  3. Things you don’t know

Focusing your research around items in bucket 2 will yield the highest impact insights. This where you are making assumptions and placing bets.

Additionally, you should seek answers to questions like: Why does a user need a product for __? What are their goals? What motivates them? How are they achieving their goals today without this product? What challenges are they experiencing?

Here are some great methods for researching big picture questions:

  • In-person user interviews. Pack these sessions full of open-ended discovery questions to help you uncover users’ goals, attitudes, motivations, and pain points. Look for opportunities as users describe their experiences. You’ll get the best results by conducting these in person, but they can be done on the phone/skype as well.

  • Observational walkthroughs. Watching users performing tasks or actions related to your problem space will provide insight into current pain points and opportunities. If you can, conduct these observations in the field. That is, observe the user in the actual environment where they perform these tasks whether it’s their home, office, or standing in line at a coffee shop. Depending on your product you may need to go out in the field to get an accurate picture.

  • Diary studies. An alternative to observational studies, in which users keep a daily log of actions related to your problem or product space. You can include question prompts for them to answer each day and also conduct a follow-up interview to dig deeper into their recorded actions. Like field studies, these can uncover what users do in their natural environments, rather than in a lab setting.

I generally shy away from online surveys as a standalone method, because it only shows you one side of the equation. You tend to get the what, but not the why. However, surveys can be a great way to recruit users, performing double duty by gathering data and screening participants for interviews. Then you can use the results of the survey and interviews in combination.

Depending on whether you have a product launched or not, you may be recruiting users, potential users, or even people using your competitors products. You can screen participants based on broad attributes (e.g. own an iPhone), but try not to be too selective. It’s particularly helping in persona development to test with a diverse sample of users so you can identify a broad range of behaviors in your product.

###Persona Development

Now that you have insights about your users and/or potential users you can develop personas. Personas are useful in helping determine what your product does and how it works, but they are also a great communication tool to align the team around common user-centered goals. We used to have a saying at Miso whenever we were having trouble making a decision: “What would Social Serena do?” (Social Serena was our target persona). This always helped us refocus back to the user.

To create personas, compile a list of behaviors exhibited by all your participants and identify cohorts of users who have similar sets of behaviors. From there you can create narratives for each persona describing their behaviors and goals, along with the context in which they use your product in their everyday lives.

To identify your target or primary persona you should look for the user whose needs are best met by your product and is a large enough population to serve your business goals – are you building a niche product or are you trying to capture the entire population of mobile users? A common pitfall is to choose the persona who is most similar to yourself, or your team. Being mindful of this pitfall will help keep that tendency in check.

You should focus first on one persona and build a product they will love. If secondary personas emerge who are served by your product that’s fine, but never design for secondary personas at the expense of your primary persona. If you try to build a great user experience for five different types of people, you’ll end up building something that is mediocre for everyone.

##Stage 2: Product Development

Now that you’ve identified your target persona and user stories, you’re ready to design product solutions. Typically, there are many design paths to the same goal. Here are some user research methods that can inform your product design direction.

  • Storyboard testing. If you’re still narrowing down the right set of use cases, storyboards are a great way to solicit feedback from users. Take a piece of paper, fold it in half 3 times until you have 6 squares of space to work with. Sketch out your user stories in 6 scenes, comic book style. Include context of what the user is doing and how the product helps. Create several storyboards and use them to discuss and compare the different user stories with your participant.

  • Card sorting. This is a great research method if you have a lot of elements in your product that need to be logically organized. Give users a set of cards with various terms written on them – actions and objects in your product. Ask users to group the cards into categories that make sense to them. The results will be invaluable to informing your information architecture and navigational design of the product.

  • Compare multiple designs. Put your three best ideas in front of users and get their reactions. These should be sketches, wireframes, or low fidelity prototypes (keynote is a great tool for clickable prototypes). Have users compare and contrast the ideas, describing different situations in which they would use each.

  • Scenario-driven testing. Once you have working prototypes, you can use scenario-driven testing to validate whether users are able to accomplish the defined user stories in various scenarios. This will validate the effectiveness of user flows in the app. Pay attention to where users get confused or distracted, and where they go down unexpected paths. Have them explain out loud what they are doing.

At this stage of research you probably want participants who are similar to your primary persona, although I still find it helpful to throw in a few outliers types as there is always something interesting to learn from different behavior patterns. An easy way to screen participants is though a short survey (try Google Forms or Survey Monkey), post the user testing opportunity (Craigslist and TaskRabbit work well), and then contact individuals who fit your criteria.

##Stage 3: Optimization

If you are already doing user testing, it’s probably some sort of usability testing. This typically happens in the final stages of product design and development, so just make sure you’ve also done the upfront research to build the right product in the first place.

When it comes to optimizing, the devil is in the details.

  • Usability testing. This is all about optimizing the user experience by eliminating error and improving efficiency. Do this before you ship! Usability testing typically involves task-based testing, similar to scenario-driven testing but at the individual task level rather than goal level. Ask users to complete explicit tasks in the product. Pay attention to areas of confusion or frustration. Ask them to talk out loud as they go.

  • Live-testing. Once your product is out in the wild, live testing is a great technique for web products, enabled by tools like Ethnio and These tools grab users in the moment they are using your product and recruit them to an interview or a screen recording of their actions. The biggest benefit of this method is connecting with the user at the exact moment they are using your product – the user’s goal and intent are authentic and top of mind.

  • A/B testing. This is the first data-driven research method I’ve suggested. A/B testing works well for making incremental improvements to visual design, interaction design, and copy. When you have enough data for statistically significant results, it also can be effectively used to optimize user actions (e.g. clicks on a signup button). Optimizely is an easy to implement A/B testing tool with good analytics. If you’re looking to test different designs that aren’t live in your product, allows you upload screenshots and ask users a series of questions after viewing them. It’s great if you’re short on time and need feedback on just one thing.

Since I opened the door on data-driven research, I should also mention the value in gathering quantitative metrics. Quantitative research only shows you half the picture, but the advantage is that it’s definitive and unbiased. Use data to understand what users are actually doing in your product and conduct user studies to understand why they are doing it. Popular tools are Google Analytics, Mixpanel, Flurry, Kissmetrics, and Crazy Egg.


###No matter what you learn, you can’t go wrong.

Consider two possible outcomes:

  1. Some of your convictions were wrong. So what. You learned something new that helps you evolve your thinking and build better products. Better to figure it out sooner than later.
  2. Some of your convictions were right. Now you have more data to support your convictions, and you know what this data helps you do? Raise money. Build credibility with press. Secure partnerships. Hire people.

It’s a win-win, and it’s never too late to start.

###Get messy

It’s also worth mentioning that the term research has an undeserved bad reputation. It conjures up feelings of effort and labor. It’s associated with rigorous academic study, not with agile startup methods.

The truth is that research can be fast and easy. It’s ok if it’s messy. The results don’t have to be statistically significant or documented in a report to be useful. I’ve offered just a handful of suggestions for various stages of product development, that can all be done accomplished in just a few hours a week. I didn’t provide step-by-step guides of how to execute each of these methods because there are plenty of available sources for that, some of which I’ve linked to below.

###Always be researching

Remember, user research is not just usability testing or a one-off project. It’s an ongoing process that should be baked into each stage of product development. Moreover, these stages are cyclical or iterative, rather than linear. This concept is mirrored in other product development strategies such as Steve Blank’s Customer Development Methodology.

Blank’s Customer Development Insight Cycle

You can move fast and break things, but it won’t do any good if you’re moving fast in the wrong direction. User research will help you build the right thing and that is the difference between failure and success.

Many thanks to Sabrina, Somrat, & Matt for reviewing drafts of this post.

You can follow me on twitter @katiesmillie.


Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems by Steve Krug

About Face 3: The Essentials of Interaction Design by Alan Cooper

The Startup Owner’s Manual: The Step-By-Step Guide for Building a Great Company by Steve Blank

Quick N Dirty User Research Workshop by Google Ventures

How User Research Methods Can Change to Better Accommate Mobile Products