Thank you!

If you’re skeptical about the value of qualitative market research, Jesse Caesar gets it. “I started out as a creative. I saw market researchers as these killjoys who came in and lifted their leg on all of my beautiful work to make it bland and palatable for the masses,” he says. “Obviously, as a market researcher, I think a little differently now.”

Caesar’s career has given him a keen perspective on products and the role they play in customers’ lives. While working at bigger agencies, he conducted research for clients like PepsiCo, where he immersed himself in the world of young athletes to study how they incorporated Gatorade in their training regimens. Now, as an independent consultant, he’s collaborated with big brands like Amazon and startups like Creative Market — and he’s ready to give the research tactic the rebrand it sorely needs.

“Qual isn’t for watering down ideas. It’s for inspiring them, vetting them and optimizing them,” Caesar says. “From my experience working with startups, I think qual is misunderstood and, consequently, not used often enough. Founders might worship at the altar of data, but they have a lot to gain from leaning into qual as a generative resource. Good qualitative market research sparks insights and adds fuel to the creative fire.

In this exclusive interview, Caesar gives a comprehensive overview of qualitative market research and the kinds of questions that qual, uniquely, can shed light on. He outlines practical advice for how startups can choose the right methodologies to suit their product and set up their research for success.

QUALITATIVE RESEARCH: WHAT IT IS, WHAT IT’S NOT, AND WHEN IT CAN HELP

For those exploring the world of market research, qual can seem like an inscrutable black box, vulnerable to misinterpretation. Here, Caesar draws out key differences between qual and its close cousin, quant, and debunks qual myths — making the case for how qual can help bring into focus potent solutions to startups’ most complex questions.

What it is: Quant vs. qual

One reason for the muddled perceptions is the fact that market research is a coin with two sides: quantitative and qualitative. “Both are valuable lenses into the product and the market, but they’re tools that do fundamentally different things. Founders often aren’t sure which to use and when,” he says.

Caesar’s here to set the facts straight, starting with quant: “Quantitative market research work draws projectable insights from a statistically significant sample. Think of online questionnaires, satisfaction surveys, or A/B testing. It’s ideal for when you have a defined set of questions you want to ask and variables you want to test for,” he says. “It’s a helpful tool for answering questions such as: How much is the customer willing to pay? If you’re trying to choose between Options A, B and C, go for quant. If you need numerical validation, and you’re working with clearly delimited metrics, don’t waste time and treasure on qualitative research,” says Caesar.

But especially in the realm of early-stage startups, those metrics and options aren’t always so clearly cut or readily available. “Not all problems are quantifiable,” says Caesar. “For those more nebulous questions, like Where does my product fit into a user’s life? you need to venture into the territory of qualitative research.”

If you want to know what your target is doing or how much, then go for quantitative research. But if you want to know why they’re doing it, or why they believe what they believe, qualitative research can get you that depth of perspective.

And in Caesar’s experience, startups don’t make that trek often enough. Founding teams tend to prefer digging into hard numbers and drawing out trend lines that can be easily displayed on a pitch deck.

“I’ve seen a bias toward quantitative studies. Technical founders especially are hungry for projectable, statistically significant data to inform decisions,” says Caesar. “But data only gets you so far. You could run an A/B test on your new positioning, but that won’t give you a sense of why a user preferred A over B. With qual, what you’re giving up in statistical certitude, you’re gaining in the opportunity to carve out a more nuanced understanding of your category, the problem you’re solving or how your product is resonating.”

In short, qual scratches a different itch. “Companies talk a big game about how much they care about their customers and their problems, but it’s often in the abstract. Qual is a way to float that customer focus up to the surface. It’s a tool that allows you to encounter your users in the wild — not just view them as a hypothetical user or aggregated persona,” he says.

“Qual is about filling in the blanks. It’s exploratory. It’s about emotions. And since it seems squishy and hard to scale, it’s easier to write off,” says Caesar. “But startups do that at their own peril. We build better products and solve bigger problems when we keep real people in mind, and you just can’t get that without qual.”

When it comes to market research, quant can only tell you so much. Data without insight is deadweight.

Here, Caesar rounds up the differences between quant and qual to clear up the path founders should embark on when they come to this fork in the market research road:

What it’s not: Qual myths, busted

Since qual deals in the domain of indistinct metrics, it’s also vulnerable to misunderstandings. Here, Caesar knocks down the most pervasive misconceptions that keep founders from taking advantage of qual.

How it can help: The knowledge gaps that qual can fill

Whether a startup is in the embryonic stages of development or refining a well-established product, here are some of the questions qual is uniquely suited to explore:

THE MANY FLAVORS OF QUALITATIVE RESEARCH (NO, IT’S NOT JUST FOCUS GROUPS)

With a better understanding of where qual spikes as a solution, the next step is to set up a study. “The methodology you move forward with depends on your research question and your product,” Caesar says. “Each one offers a distinct lens on your problem; sometimes, you’ll have to use a combination of them."

Here, Caesar explains each research tool and the observations they’re designed to illuminate about the user.

The focus group

“When people think about qualitative research, they usually picture focus groups with stereotypical, stuffy settings,” says Caesar. “But in reality, you can mine some of the richest insights from group dynamics.”

In Caesar’s experience, a typical focus group lasts about two hours, with about eight people in a room and a moderator guiding the conversation. “Choose this format if you want feedback from multiple people on a particular task or concept that isn’t context-dependent, and you want to observe results from a controlled environment,” he says.

A focus group might be ideal if you want multiple subjects to interact. “That group setting can reveal important frictions, or the idea-building that leads to strategic insights,” says Caesar. “I’ve moderated groups where I divided the room into two teams, and I assigned them to develop their ideal product and sell it to the opposing team. That helped the client identify important criteria for their own product, as well as the right language to describe it with.”

Startups are sometimes wary that focus groups would encourage groupthink — that participants in a group would conform to a “hive mind” rather than share their own opinions. “A skilled moderator knows to preempt this,” says Caesar. “I’ll have respondents record their ideas or reactions individually on a piece of paper before we discuss it as a group. Once we’re in the group setting, I make it absolutely clear to participants that there are no right or wrong answers, and that they should feel free to disagree with one another so long as they express what they honestly believe.”

The in-person, 1:1 interview

“This is the best option for when you have a lot of stimuli to share, whether that’s written concepts or a digital demo, and you want to capture nuanced reactions,” says Caesar. “Or, if you need to speak with experts or busy professionals whose schedules are tough to pin down.”

This is also the best method to start with if your questions concern a sensitive topic. “Are we talking about health, wealth or sex? In a private setting, face to face with a human who’s earned their trust, they’re far more likely to open up,” he says.

Because of the individualized attention, the 1:1 takes more time and money to execute. “Make sure there isn’t a more effective way to address your objectives,” says Caesar. “No-shows are a lot more disruptive to your study, so be sure to book more 1:1 interviews than you think you might need. Double-book with floaters to cover your bases.”

The ethnographic study

Ethnography is ideal if you want to observe your target interacting with the product in their natural habitat, typically at a consumer’s home, their place of business or school, or during a shop-along at a store in their area,” says Caesar.

“A conversation in the user’s context often yields the most striking findings,” he says. “You can pick up on body language and intonation, and meet them in a place where they’re comfortable, surrounded by ‘artifacts’ that can help illuminate your findings. If you have the time and money to expend on doing your interviews in-situ, then you’ll get the closest view on your target.”

Take extra precautions when you’re in others’ spaces. “Since you’re going to be on someone else’s turf, be respectful. Get permission ahead of time if you’re going to be filming, and be mindful of house rules.” Also take care to avoid overwhelming the participants with too many observers. “I like to limit the ratio to four observers to one participant, or only two participants if I’m there with a videographer,” he says.

As an example of how qual can uncover unexpected insights, Caesar recounts an ethnographic study he did for an insurance company looking to better understand and how to more effectively connect to a segment of potential consumers based on an algorithm of psychographic data. After spending time in customers’ homes, he was surprised to discover that a significant portion of customers in this segment were avid collectors.

“I’m not just talking about a few figurines on the shelf. It was more like houses that were filled entirely with a singular obsession,” says Caesar. “One woman had two pugs, plus little pug statuettes outside of her house, bookcases filled with pug plush dolls, pug photos, pug books, and urns of pugs past. When I used her bathroom I looked up and staring back at me was a watercolor of a colonial era scene with a pug riding a horse through a pug populated town, titled ‘Pug Revere.’ This unexpected insight about how so many customers were collectors led the home insurance company to come out with a new collectible coverage product offering. It’s just another example of how ethnographic work can help you uncover a variable you wouldn’t have even known to ask about in a survey.”

Sure, qual is a form of market research. But at its core, it’s storytelling: you’re learning, in the most intimate way, the story between the user and your product.

If resources are tight, scrappy startups can adapt ethnographic findings to a more budget-friendly form. “Let’s say you can only afford a remote, video chat interview. You might ask your research participants to take a video of themselves performing a task at home. Sometimes, a supplemental element like that will suffice,” says Caesar.

Phone or live video interviews

While they’re physically removed from the subjects, phone and video chat interviews are a low-cost way to dialogue with numerous users. “This is your most useful tool if your respondents are widely dispersed geographically, when you’re trying to schedule with busy people, when you have a limited number of respondents to interview — I’d say 12 or fewer,” says Caesar. “It works best if you’re interacting with people 1:1. Live, remote group interviews can be noisy and hard to facilitate, so I wouldn't recommend them.”

There are a few things you can do to help make up for the physical distance or the lack of visual cues. “Use a good web-based tool if you want to share stimuli for reactions, or read facial cues,” says Caesar. “This should include a tech check for your moderator and participants ahead of time, screen-sharing, recording, archiving and a virtual ‘backroom’ so you and the moderator can communicate during and in-between interviews.”

These interviews worked particularly well when Caesar was tasked with interviewing for an international wellness brand. “They wanted to talk to their own distributors at various levels of seniority, based in different locations, about a few positioning concepts,” he says. “I was able to show them the different concepts on a screen and talk to them about how each might impact the product’s success.”

Online asynchronous discussion board

“With this method, you’re posting questions about your product online, and people sign on at different times of day to respond. This is best suited for studies where you have a very wide geographic dispersal of targets,” says Caesar.

“Even though you’re removed from the action, you still have the opportunity to follow up with responses and ask for clarification,” he says. “But you won’t see any epiphanies, or see the assignment in progress. To mitigate the distance, you might try to ask respondents to upload videos of themselves as they complete a task, or take pictures and share them.”

Jesse Caesar

RAMPING UP QUALITATIVE MARKET RESEARCH FOR SUCCESS

Once you’re on board with qual and have chosen a format, you can get started with revving up the research project. In Caesar’s experience, startups all too often putter into the same pitfalls, from a lack of focus to projecting biases onto the findings.

To get the most out of your research, follow Caesar’s checklist for effective qual studies.

Keep your eye on the target.

Before you dive into research, start with a basic grasp of the problem you’re solving for. “Don’t do research if you don’t have a problem or question. Qual is not the hammer in search of a nail,” says Caesar. “You’d be surprised at how often companies propose doing research for the sake of research, without any reflection on what, specifically, it will help them solve. Your objective directly impacts every aspect of the research, from the scope of the study down to the questions you ask. Spend the time upfront to define what you want out of it.”

For example, a strong objective might look like this: Our objective is to investigate what design and copy elements grab attention as well as motivate purchase in order to optimize package design and drive preference at the shelf.

“A good objective statement includes an ‘in order to’ statement. You should know exactly what this research will entail for the business,” says Caesar. “This will keep the work that follows, including the implications and recommendations in a final report, relevant and responsible.”

Plan for execution — but get realistic about your resources.

Caesar breaks down three options that startups can explore to carry out their research:

Consider the resources you have before investing in high-quality qualitative research. “Most larger agencies won’t work with you for anything less than $30,000. A freelancer or a really limited study might cost you $12,000 to $15,000,” Caesar says. An early-stage startup can still find ways to work around a shoestring budget. “Instead of hiring a vendor, reach out to the best-connected people on your sales team to come up with a list of targets. Find some way to scale your research down so that it’s affordable, but doesn’t compromise on accuracy or quality,” he says.

Leave your expectations at the door.

“One pitfall I encounter is that people enter into market research with a preconceived idea about the kind of outcome they want to see. Confirmation bias blinds you to those nuances in human behavior you’re there to observe,” says Caesar. “You can go in with a hypothesis, but you have to be okay with it not holding up.”

While all data is susceptible to being misinterpreted, qualitative research prompts an extra layer of caution. “With something as nuanced and observation-driven as qual, it’s all the more important to be diligent about not interpreting the results through the prism of your own agenda,” he says.

You need to come into qualitative market research with an absolutely open mind. If you’re fixed on a certain outcome, you’ll selectively read the output in your favor. The point of research is to be humbled by it — and inspired to do better.

In fact, qualitative research is at its most valuable when its insights subvert expectations. “I was doing some early ad testing for a client, and at first, they were apprehensive that one creative approach would alienate consumers. We tested it anyway just to see, and to their surprise, there was no pushback at all. While their fear was reasonable, when we actually looked under the bed, the monster didn’t exist,” Caesar says. ”If they hadn’t tested against their expectations, they might have gone to market with something that wasn’t loud enough.”

POWER UP YOUR EXECUTION WITH THESE SIX PRINCIPLES

Once you’ve decided on a methodology and set up your study, lean on these principles to maximize the impact of your research and tailor your discussion guide.

1. Craft questions with care.

Questions are a researcher’s most potent tools when they’re crafted thoughtfully and wielded adeptly.

He cautions companies from overloading respondents with questions. “The temptation’s understandable. You’re getting this unprecedented access to people’s inner psyches, so why wouldn’t you ask as much as you can?” says Caesar. “It’s all fun and games until you realize, after a two-hour long session, you’ve only gotten shallow answers to scattershot questions.”

When you’re conducting market research, you only have a limited time in the room with respondents. Whether you have fifteen minutes or three hours, use them wisely. Craft crisp, impactful questions to make sure you don’t walk away empty-handed.

“There’s no hard and fast number of questions you should ask,” says Caesar. “But you should be extremely methodical about crafting them. Strike the balance between breadth and depth.”

2. Empathy is a moderator’s secret weapon.

“In quant, you calibrate the length of the surveys to avoid fatigue, so people aren’t just clicking to get through it or dropping off en masse. The same thing happens in qual. People will start shutting down if they’re not engaged in the process,” says Caesar.

Start by creating an inviting atmosphere where subjects can feel empowered to be candid. “Before kicking off a session, I ask participants to do two things: Be selfish, and don’t think too much,” says Caesar. “Communicate that what they’re about to spend their time on is going to make a real impact on how a company builds a product for them. They share whatever thoughts organically come to mind.”

Only after establishing discussion norms should you ease into the inquiry. “Start by having them talk about themselves,” he says. “Sometimes they’ll talk about their kids or what they’re binge-watching on Netflix. Before you get into your agenda, let them tell their own story.”

If people are disengaged or uneasy in focus groups, they’re going to clam up and not give you the goods. On the flip side, when they see you as a welcoming, authentic human being, they’ll open up and start to dish out the truth.

Conducting a focus group with empathy requires a deft hand. “A good moderator will not only be able to make respondents feel comfortable and heard, but also make sure that there’s no one person whose strong opinion is swaying the group,” says Caesar. That doesn’t mean, though, that the moderator is the star of the discussion. “If your moderator’s talking as much as the respondent in the room, well, you’re going to end up with more data on the moderator than your users,” he says.

3. Always be willing to suspend the agenda.

“One of the most powerful things about qual is that it’s fluid and open to improvisation. Don’t passively run through a list of questions. If you feel that a set of questions isn’t getting you anywhere, recalibrate, adjust," says Caesar. “You can’t do that in a static survey.”

Of course you should come with a discussion guide prepared, but if you start hearing something that’s interesting or strange, follow it. Explore that path. Don’t be afraid to go off script — that’s where the magic is.

“Once, during a two-hour session, I had allotted 20 minutes for a series of questions. The respondents had brought up something unexpected, and we ended up taking 45 minutes. For the rest of the interview, I just had to pare down some questions here and there, to make it work. But it ended up being a worthwhile tangent,” says Caesar.

4. Triangulate the truth.

Caesar picked up this approach from his mentor, Andy Greenfield. “Triangulating on the truth means keeping the same set of questions you’re trying to resolve, and bringing in different formats and exercises to try to answer them. There’s not one ‘perfect’ way to ask a question, because people think and express themselves differently. Diversify your pathways to make sure you eventually arrive at the truth,” he says.

If your first approach isn’t working, switch up the medium. “Give your respondents a task where they have to draw a picture, then talk about it a bit. You’ll often find that it spurs something new,” he says.

On a higher level, triangulating might involve a mix of methodologies to access a more complete set of insights. “A farming startup I worked with wanted to understand what motivated their shoppers to gravitate toward certain kinds of produce,” says Caesar. “In focus groups, we had surfaced a lot of insights on what customers cared about in terms of the hierarchy of different brand attributes and benefits. But we unlocked so much more about the decision-making process when we actually hit the grocery store in an ethnographic study. By observing some of those same respondents, in a different setting, we got to see those insights in action, right in the produce aisle.”

5. Sleuth for the patterns.

“Remember that qual won’t give you hard, fast numbers. Qual gives you directional insights in the form of patterns,” says Caesar. To keep your team alert to the patterns that qual exposes reflect on the research as you conduct it — don’t wait until afterward. “Qual isn’t about researching once, then delivering a report a couple of weeks later. It’s an ongoing process. When I’m facilitating studies, I like to leave time after each interview to have a quick check-in with my team and talk about what we’re hearing,” says Caesar.

“After all the research is done, take a step back. You might notice that people have been hinting at the same issue, or same type of issue. Maybe they’re having similar reactions to something,” he says. “Connect the dots, and look for the constellation in-between.”

One such pattern eventually developed into a restaurant, at an iconic destination. “The Cosmopolitan of Las Vegas had an empty showroom, basically a blank canvas, and wanted to create a compelling, different experience on the Strip,” Caesar says. “I interviewed frequent Vegas visitors, and we discussed what motivated their trips, what they felt was missing, and what might be interesting and novel to them. Consistently, we heard that Vegas experiences were feeling stale and cookie-cutter, that there weren’t enough opportunities to feel special and fancy. As a result of those interviews, they came up with the concept of Rose. Rabbit. Lie., which combines the excitement of clubs with the intimacy of lounges.”

6. Garner alignment from the get-go — and sustain it after the research ends.

“A kick-off meeting is absolutely crucial. Before you start reaching out to vendors, make sure that everyone who would be impacted by the research is clear about the problem, the point of friction and the agenda. Get people invested, and keep them involved throughout the research process,” Caesar says. At minimum, get alignment from decision-makers from marketing (if the study is communications-oriented) and product (if it impacts development).

Market research is all about garnering different perspectives, right? But what people often forget is that successful market research starts from within.

After all, once the focus group has disbanded and the easel board is put away, you’re left with a final task: to share that trove of knowledge with your team.

“At the end of the day, the research won’t mean a lot unless you socialize your results. If you’ve gotten everyone on board from Day 1, they should be excited to draw from those findings and tinker toward new directions and solutions,” says Caesar. “That’s the moment where qualitative market research really proves its worth —  the moment when imagination is ignited and sparks start to fly.”

Photography by Michael George.

First Round ReviewThe best advice in tech
Subscribe for Exclusive Access