Thank you!

Good intentions don’t work.”

This is one of the most important things Anurag Gupta has learned from working at Amazon, where he is currently General Manager of Amazon Redshift and Amazon Aurora. It’s a counter-intuitive notion — after all, we all believe it’s our employee’s passion and energy that lets our companies accomplish great things.

“The problem is, your employees already have good intentions,” says Gupta. “No one comes into work and says, ‘Today I’m going to push a change that’s going to break the site,’ or ‘I’m going to hire someone I know I’ll let go in six months.’ But these things happen. Good intentions don’t fill the gap between what your employees want to accomplish and all the mistakes that inevitably seem to occur. And exhorting them to do better or focus harder won’t make a difference to the number of mistakes in the long run. Your employees already want to do the right thing.”

So, if good intentions don’t help you reduce mistakes, what does? At Amazon, Gupta learned that the answer is mechanisms — processes that are repeatable, measurable, auditable and improvable.

At First Round’s last CTO Summit, Gupta showed a number of mechanisms that could be applied to the hiring process to steadily raise the bar for new employees, improve candidate experience, reduce hiring mistakes and save time — which becomes increasingly critical as you scale.

Raise the Bar Hire By Hire

“A lot of companies say, ‘We only hire people better than ourselves.’ But what does that actually mean?” Gupta asks. “If you try to mechanize it, you end up with a forest of questions: Better than whom? The best person? The average person? Better at what? How does one calibrate what better even means?” At Amazon, he learned how a bar raising process can help you build a stronger company with each hire.

Gupta recommends assigning people already within your company to act as bar raisers. Include them in every hiring conversation and on every interview loop. Empower them to determine if someone is an improvement over existing talent or not. “Look for people with high standards and good judgment,” says Gupta. “Tell them their job is to ensure that somebody is going to be above the 50% bar of the folks already at your company for that role at that level.”

It’s critical that the “bar raiser” does not feel the same pressing need to fill a role that the hiring manager does. They should come from outside the hiring team but know enough about the role to judge talent. You don’t want them to be influenced by timing; you want them to focus on performance.

“Your bar raiser should also run the debrief after every hiring loop,” he says. “Everyone gets together and shares feedback they’ve written up beforehand. This ensures their initial feedback isn’t swayed by others. The bar raiser is also responsible for ensuring everyone’s voice is heard and the hiring loop moves towards consensus.” At Amazon, the hiring manager and the bar raiser must both believe the candidate can do the job and exceed 50% of the folks currently doing so.

And, as with every good mechanism, there should be alternate paths built in. “If the hiring manager disagrees with the bar raiser on a candidate, you should know what the escalation process is,” says Gupta. “In my experience, this is really rare, but when building any process, you need to know how to deal with the rare exceptions.” At a startup, you might bring the decision to one of your founders.

Build a Hiring Funnel

You also want to make sure your bar raisers are consistently effective. "Look at the people you’ve hired who wash out within a year and see whether they cluster around any particular bar raiser," he recommends. “You can remove more and more variability from the hiring process by making sure your bar raisers are people that have great hiring judgment. And they’ll only get better as they do more interviews and gain more experience.”

A key metric Gupta uses to ensure his managers are maintaining a high bar is the ratio between first phone screens and offers accepted. “Across your teams, you’ll be reviewing thousands of resumes and conducting hundreds of phone screens each year,” says Gupta. “That gives you good data across teams and ensures each of your teams is maintaining a high bar.”

You want to be very selective about the people you choose to engage with further along in your hiring process. “For every person making it all the way through the interview loop, that can be 20 person-hours,” says Gupta. “If you multiply that by your hire ratio, you’ll see that this is just not scalable. You also have to consider the candidate experience — they're investing time with you just as you are with them.”

If someone makes it all the way to an onsite interview, you want to be able to tell them 'yes' a fair percentage of the time.

“You might institute a process where, after phone screen one, you eliminate a third of the people you talked to, and another third after phone screen two,” says Gupta. Being rigid about this prevents you from inviting too many people for in-person interviews, a significant step up in time and energy, especially if you have several of your existing employees talk to them.

There are two possible mistakes you can make — you can either hire the wrong person or fail to hire the right one. None of us like to say no, but you should bias toward the latter. And doing so early in the process, even with limited data, saves time for everyone, including the candidate. “We tested this. We had some ambiguous phone screens and let the candidates through to onsites. None of them made it through to offers. Hard as it is, we’ve learned to be more disciplined.”

To fairly and evenly weed out candidates — and protect your employees’ time — Gupta also recommends assigning a take-home coding test between phone screen one and phone screen two.

“This assignment should be a trivial coding problem,” he says. In the past, he’s assigned something that would take a moderately skilled coder 10 minutes to finish. “What’s crazy is that even if you assign a problem like this, one that’s trivial, around 30% of people will fail — I couldn’t believe it at first. But by now, we have hundreds of data points and the numbers hold. If coding is what the job is about, candidates have to be able to show they can do it, particularly outside the artificial environment of coding on a whiteboard during an interview.”

This same technique can be applied in other functional areas. For instance, your communications team could ask a candidate to edit a press release for grammar and clarity. Basically, you want to see if the person can perform on some of the core requirements of the job outside the artificial stress of an interview.

Don't Go Easy on Culture

Of all the dimensions of a hiring process, determining culture fit seems like the hardest to standardize. And, if you’re doing a rigorous job bar-raising and running your interviews, a lack of culture fit may represent half of the folks who wash out after a year. "Half your onsite interview time should focus on culture fit,” Gupta says. This means you need to be crystal clear about what your company’s culture is and what it’s not.

Amazon took the time to figure out what its core leadership principles are and put them on its website. They think about what these principles mean for every job at every level.

Prior to joining Amazon, Anurag Gupta served as Vice President, Engineering at Oracle.

Also, if you dedicate a good chunk of your job listings to cultural expectations, people can self-select whether they’ll fit in. “Part of why you start your own company is because you want to create the environment you want to be in,” Gupta points out. This is perhaps the best argument for enumerating and mechanizing cultural values from the start: Culture is made by the people in it, and you have the opportunity for every new hire to strengthen the environment you’ve envisioned.

Part of this is also making sure existing employees who sit on hiring committees have internalized the touchstones of your culture. That way, they can be better judges of whether candidates exhibit those qualities.

“Use the precise language you’ve decided on in your one-on-ones with existing employees. Use it to describe how they could be better and where they're doing great,” says Gupta. “We’ve made it part of our annual review process. People’s performance against these values is one of the key things we look at.” Using a standard lexicon also helps your managers calibrate with each other on what each of your principles mean for each level and role.

He also recommends taking the extra step and pointing candidates to your list of core values before they show up for their onsite interview. “Say, ‘Look, this is what we care about, and we’re going to ask you about some of these things,’” he says. Then talk to your hiring team and assign each interviewer a different cultural competency to ask about. “That way you’re not all asking them about one thing versus another.”

In this vein, Amazon has made its core principles publicly available, framing each in terms of good leadership:

These core leadership principles are another way Amazon enumerates and mechanizes its cultural values across the entire company. This way, the culture they represent and instill in new employees, remains just as strong now as when Amazon was a startup.

Recommended Articles
First Round TalentAccess exclusive opportunities at the startups in our community