Thank you!

Marco Rogers loves interviewing engineers. Here’s a quick back-of-the-napkin showing just how much: he’s been an engineering leader and hiring manager for the last seven years. Over that time, he’s hired over 80 engineers. To fill each of those roles, he’s personally interviewed approximately 5 candidates on average after initial phone screens. That’s at least 400 interviews, or an in-person interview every work week for seven years — and that’s not counting time dedicated to prep, screening, references or debriefs. Most impressive is that he’s designed a system that has gotten his technical teams involved — and energized — to interview their future colleagues. To him, interviewing is a team priority: engineers on his team who are seasoned interviewers can conduct 12-16 interviews per month.

For those who’ve seen him in action, it’s inevitable that Rogers ended up at a recruiting software startup like Lever. Every engineering leader spends time on building and rebuilding teams, but while at Clover Health and Yammer, Rogers obsessed on designing and refining an immaculate interview process. It was that craftsmanship that helped him hire over a dozen developers at Yammer, grow the Clover engineering team from one to 50, and start to scale up Lever’s technical team.

In this exclusive interview, Rogers starts by debunking some of the common recruiting tropes, explaining why they are outdated or misleading. Then he spotlights his top four interviewing practices — and how they fit within his broader recruiting methodology. Lastly, he recommends the first, low-hanging change that startups can make to retrofit their interview process.

THREE HARMFUL RECRUITING REFRAINS AT STARTUPS

THE REFRAIN: “We test the technical knowledge and skills needed for the role in the interview.”

THE REALITY: You don’t actually know what you need. You’re figuring out what you need.

Across the startups he’s joined (and those he hasn’t), Rogers seeks to change the narrative around mapping relevant knowledge and skills to role. “Even if you’re a serial founder or seasoned technical leader, I’d still argue that you don't actually know what you need in the early days of a startup. There are so many variables, and the equation changes as the market, competitive landscape, customers and other elements shift,” says Rogers. “In the tech startup context, what you’re actually doing is figuring out what you need. You don't start with the answers; you work toward them. Instead, people orient around static sets of skills. So you can recite what it means to build a B+tree. That might be helpful, but not the entire picture.”

I want to make a stronger statement: the skills that people are hiring for are not the skills that they often most need.

So what’s at the root of this misconception? “When I’ve dug into conversations with people about it, I’ve realized that they haven't actually decided that interviewing and hiring is a critical business activity,” says Rogers. “They're kind of just trying to get through it. And because they want to get past it, they look for and lean on patterns, opting to repeat what others have said or done. Of course, they care who they work with, but it’s more about getting people in the door. Then they'll figure it out. That's the attitude. Hiring is not the real activity; it's getting people. It’s a subtle difference that short circuits process for outcome.”

THE REFRAIN: “With engineers, only technical skills truly matter. Soft skills are a nice-to-have.”

THE REALITY: Every engineer must be equally skillful with code and colleagues.

First, Rogers takes issue with the framing of soft skills. “The problem with that framing is that it suggests that you can’t evaluate or teach those skills — that they are inherent qualities that some people have and some people don’t,” says Rogers. “They focus on the ‘soft’ part and forget they are ‘skills’ that can be measured and cultivated in people. I put those people skills at the same level of importance as technical skills. Can you write JavaScript and communicate effectively with business stakeholders? That’s what I need in a growing organization.”

It’s the business’ end goal really makes this an obvious default for Rogers. “First and foremost, we have to address the business reality — what we're really trying to do. If we’re talking about companies that want to grow 10X every couple of years, you have to assemble a really solid team that's going to take you there,” says Rogers. “It’s hard to keep that in mind when there’s the first or next thing to ship. So you have to explicitly test writing code and working together in the interview process.”

Rogers helps his engineers strike balance between the “soft” and “hard” skills by asking them to get answers to the following questions around three key areas:

THE REFRAIN: “We’ve picked a few people to interview with them — let’s get it scheduled.”

THE REALITY: Thoroughness and consistency is paramount for optimal, unbiased results.

When Rogers talks about an interview program, it’s a crafted process. “So if you come on site with us, you're going to have four sessions. That’s our ‘right’ number, because we don’t want to ask too much time from candidates and be grueling, but we also need to have enough data points. The four sessions include three evaluations sessions with the team, each an hour long. Then there’s a session with the hiring manager, usually me, so I can answer any questions about how the team works,” says Rogers. “It’s designed this way so candidates can talk to a number of people: engineers of every tenure, so they can paint a picture of the day-to-day and their career path. We get to put you through your paces, and so candidates have several opportunities to display different skills. Each session has a fixed exercise or questions to ask, and then some time for unstructured dialogue. There’s a balance to strike here. The consistent structure and exercises help to reduces bias. While the open conversation makes it human. We’ve gotten feedback that this agenda and duration gives candidates sufficient time to start evaluating us as a potential fit.”

Lever Director of Engineering Marco Rogers

STEPS TO TAKE TO MAKE INTERVIEWING A CRITICAL BUSINESS STRATEGY

Rogers has designed a system to make interviewing a hallmark experience for candidates and a strategic advantage for your startup. Here, he shines a light on four components, but recommends using them together, so their impact compounds. Let’s get started.

Interview in three-person groups.

A go-to technique at Yammer and Clover, three-person interviews are one of Marco’s most long-standing and valued interview preferences. He saw the benefits of them early in his career, but more so, observed how traditional interviews are broken. “One-on-one interviews tend to lead to a unidirectional candidate-and-interviewer dynamic. A person asks a question, the other person tries to answer it. Rinse and repeat,” says Rogers. “We know that that’s not a great candidate experience, as they come away feeling grilled or discouraged after trying to answer a bevy of questions. Interviewers can also get disenchanted, when, instead of following the thread of the conversation, must cycle through questions like a more rigid, rote activity. From the angle of both parties, the interview can lose its more organic, collaborative aspect.”

The quickest and most effective way to move beyond that poor interview experience is to add another node: a third person. “Add another colleague into the mix as a second interviewer. I’ve found that it not only brings a higher fidelity and signal to the conversation, but also breaks down the flow and nature of the conversation,” says Rogers. “Sometimes a person will be talking to the candidate, and other times, they’ll address their colleague. I find questions and answers bouncing between three points opens up the discussion and evaluation much more.”

Three-person interviews generate more signal and meaningful meandering than traditional 1:1 interviews.

To show how powerful the three-person interview can be, Rogers’ shares a few scenarios, which he’s experienced and anonymized, that drive home the benefits of this interview approach:

It’s tempting to only tap your senior people to interview candidates. Less experienced people get a different signal — namely, if they think they can learn from and be led by a candidate.

Everybody — the entire team — interviews.

This interviewing principle is one that Rogers has seen get pushback. “For some, it’s an issue of scope and something leaders and recruiters should be doing. For others, it comes down to a concern that not everyone can represent the company in an interview. People also say that it’s too costly to have everyone trained and involved,” says Rogers. “The one most people have the hardest time with is cost. But it’s going to feel costly — and it should. Interviewing is a critical business activity. It's like saying, ‘Hey, we're having engineers write a lot of code. That feels costly.’ That's kind of the thing we need them to do, and that's how I feel about interviewing. I need the team to help me understand if we are bringing on the right people. These are people you’ll be working with every day for years. I think that’s worth a few hours of your time periodically.”

Startups are either growing quickly or want to scale. Either way, having more people being able to interview candidates skillfully and efficiently is an advantage. As the engineering leader, Rogers sees it as his responsibility to develop this skill in his team. “I put myself in a position to evaluate how the engineers on my team are doing in interviews. In the early days, I pair myself with new interviewers in the three-person sessions. In those situations, I’m not only evaluating the candidate, but observe the engineer’s ability to interview,” says Rogers. “When we do a roundup with all the interviewers, I ask each person to reflect back to the group how the sessions went. I can ask more questions, probe and help them unpack what happened. This paves the way to offer feedback and suggest other ways to get through interview challenges.”

The other advantage in having everyone interview is that it’s easier to identify members of the team who are comfortable, confident interviewers — people who the engineering leader can tap to serve as mentors to make less experienced colleagues better interviewers. “I get to know which engineer is skilled with candidates, both from feedback from the roundup, as well as being in the interview with them. For example, I keep notes on who’s good at conducting the technical exercise or a certain series of interview questions that we ask,” says Rogers. “With that context, I have an understanding of interviewers’ strengths and who can help their colleagues improve their interviewing skills as they get more sessions under their belt.”

Your entire team should conduct interviews. Everybody. If you don’t want some people to interview, ask yourself why. If you’re worried about how they’re representing the company, there’s a bigger issue at hand.

Commit to doing more interviews.

Rogers is a firm believer that initial interview screens and filters are broken. “Conventional wisdom is that you can look at a piece of paper and tell if you are looking at the top 10%. At every turn, I’ve found that to be incorrect. Talk to engineering leaders about how their team members are doing and how they thought they’d perform when they hired them. Increasingly performance doesn’t match up with pedigree. Maybe they attended an Ivy League school or worked at Google, but they're not coming through,” says Rogers. “Conversely, you have that person that's really kicking ass, but when you look at their profile, you didn't see that. You didn't anticipate how high performing they are today, because it’s about if they had the right opportunity, motivation, and investment in their work. This not only has a lot to do with the individual, but the structure that a company provides to bring the best work out of their people.

All of those elements don’t really show up on a LinkedIn profile or resume. Those things are about how good someone is at getting a job, not how a person will add value to your org. Profiles are only one data point, and you need more.”

Given that screens and filters are counterproductive, Rogers sees the only way to fix them is to do many more interviews. “From talking with my peers, most bring about two candidates on site for an interview in order to hire a mid-level engineer. I believe that there should be many more interviews—at least ten for every hire made,” says Rogers. “If you’re at an early-stage startup and you’re going to be hiring a lot, set up a structure that can handle interviewing many candidates. And by hiring a lot, I’d say at a clip of 10 or more people every six months. If you’re only hiring about five people in a year, you can get away with doing fewer interviews — but I wouldn’t recommend it. One year, I hired 50 engineers. I couldn’t have done that in a high quality way without a system that let me interview about 500 candidates. Create that interview-intensive system, stick to it, learn from it and you'll make better hires. That's my stake in the ground.”

Rogers readily admits that partnering with an in-house recruiter has been the only way he’s been able to create a hiring system that can support many interviews. “Even early on — especially early on — you need someone collaborating with you on the scale and scheduling of all the interviews for a hiring process. At my last three companies, I worked with a recruiter, a colleague whose job it was to help set up the interview program I designed and make it consistent,” says Rogers. “It’s on me to make sure that the program works the way that we're specifying today. That’s another reason why, as the hiring manager, I not only do a final evaluation of the candidate, but also get feedback on how the interview process went, and where it could improve. This is the information that helps us make the interview process better and more efficient which, in turn, helps us interview more people effectively.”

You want a top 10% developer? The most effective way I’ve found to get to the top 10% is to interview and say no to the other 90%.

Round up and huddle.

Rogers is a champion of roundups and huddles to wrap up an interview process. It’s the final piece of the puzzle, and the dynamic exchange of information that brings critical context that is rarely captured in written reviews. Huddles take coordination though. “When I talk to other people about huddles, they're not sure it’s worth it. It’s mostly because of logistics. To bring together everyone who interviewed candidates for a role is a lot: six engineers from three two-interviewer sessions,” says Rogers. “They are 30 minutes long, but once you get good at them, you can get them down to 15 minutes. I highly recommend that every team takes the full half hour if they are just incorporating a huddle into their hiring process.”

Here are the objectives for the huddles that Rogers runs:

The huddle helps bypass the biases that traditional screens and filters reinforce. It’s also a prime opportunity for leaders to shine a light on how they make decisions.

TYING IT ALL TOGETHER

From initial screens to session construction to debriefs, the way startups interview candidates needs to improve. To start, don’t fall prey to interviewing tropes. Recognize that you may not know the qualifications you need for a role, that engineering candidates must meet high expectations on both technical and “soft” skills, and that you need a thorough, standard interviewing system. That regimen should start with three-person interviews, which is not only a more effective way to evaluate candidates, but also a valuable opportunity to develop your pool of interviewers. Everyone should have an opportunity to interview — no exceptions. If you are anxious about that decision, dig into why. Commit to doing more interviews; it’s the best way to bypass faulty filters and screening. To get the top 10% you need to interview the other 90%. Finish each onsite interview with a huddle. They generate dynamic reads on candidates, and bring visibility to a leader’s decision-making.

“So what’s the first step to level up how your startup interviews candidates? There’s actually a step zero, which is asking: are you getting enough different data points in an interview process? If you need a more powerful signal, try this methodology. If you’re a startup that has its founders interview a candidate and then make the offer, involve more people in the interview process and debrief in a huddle. The highest leverage will be getting those interviewers rounding up to compare notes,” says Rogers. “When I talk to people about interviewing, I notice that a lot of companies are just still doing the interview style that Microsoft pioneered in the 90s. That’s what much of our industry still does. But that gets us 90s Microsoft people. If you’re a scrappy startup, your hiring expectations and approach must be different. This system has not only strengthened our candidate pool, but also the cohesion, versatility and diversity of my engineering team.”

Photography by Bonnie Rae Mills.

First Round TalentAccess exclusive opportunities at the startups in our community