I recently attended a small networking meetup of entrepreneurs and startup founders. About halfway through, the organizers invited people to give a small elevator pitch about what they’re working on and how they can help the group, and what help they need. As an ENFP I relish the opportunity to get up in front of people and share something and hopefully learn something in return.
So, naturally, I raised my hand. Instead of giving the smooth, polished pitch I normally give about what I’m working on, I stammered. I had trouble clearly articulating my vision and opportunity, and how I could help everyone there. In short, I choked.
Choking. Failing to, under pressure, perform something you expected to be able to do. It’s a terrible feeling.
How did that happen to me? I’ve given dozens of presentations in public – many for hundreds of people. I can usually talk off-the-cuff in social situations. Yet, in this intimate setting of 30 or so people, telling a story I’ve told countless times before, I came off as an idiot. I hadn’t been drinking (well, except a bunch of La Croix flavors I hadn’t tried yet). I wasn’t tired. What happened?
That question was still rattling around in my head when coincidentally, a few days later, I listened to the Freakonomics podcast on Choking. This spurred a bunch of thoughts on how we (as managers) build systems up to inadvertently cause our high performers to fail under pressure, and how we turn away people that would be high performers in our companies through poor interview processes.
Note, this a little bit lengthly, so feel free to jump ahead to various sections:
Your Company
Let’s consider a scenario, one that I’ve been in and you doubtless have, too. Your team has a project that is pretty important to your company. Maybe it’s something for a big client, or you had some commitment. In short, a deadline.
You check in daily with your team – are we OK? Any blockers? Remember, Friday is the deadline! This is super important! I know you can do it, you got this! If we get this done by the deadline, everyone gets a bonus!
Your lead engineer owns this project and today is the deadline. They leave for the weekend in which they have big plans (they’re getting married!). After they leave, you’re still in the office reviewing things and realize that the thing that was supposed to be done to ship this thing, supposedly done by the lead, wasn’t done. At least, it’s not in the source repository and certainly not deployed to the live site.
You consider this person your best lead, and they have a track record of primarily delivering in the past, although it hasn’t always been perfect.
What do you do? Take a minute and think about what you’ve done in the past, and what you’d do now.
Chances are, if you’re like most people, you’ll be upset – this person reflected poorly on you and your team’s reputation. Maybe the client called you up angrily, or the stakeholder wasn’t pleased. You may think that you’ve got to come down hard on this person. Institute more check-ins. Let’s do a twice daily standup! Oof.
If you’re like many managers I’ve worked with, your natural, intuitive, first desire is to clamp down on the individual to make sure, by gum, this never happens again. It turns out, this behavior, although feeling intuitively correct, is actually the opposite of what you should do.
Let’s learn why.
My Childhood
I remember a fun thing my mom would do for us kids when we were bored and needed something to do. It’s a simple thing, but when I was little, apparently seemed to be enjoyable.
Put 2 cups of corn starch in a big bowl. Add about 1 cup of water, slowly, stirring the cornstarch so that all of the powder is wet. Continue adding, and stirring slowly, until you have added all the water, and you have a syrupy consistency.
Now, the fun part. Take your measuring cup, and quickly and firmly hit the cornstarch mixture with it.
Trust me.
Instead of the expected splashing of this sweet-smelling mixture all over your kitchen, it will firm up – like a rock. If you slowly move your finger through it, it’ll feel like quicksand. But if you scoop this stuff up and work it into a ball, putting pressure on it, it’ll remain spherical. When you stop, it’ll melt back into the bowl.
What’s going on here? Turns out, this is a demonstration of a non-Newtonian fluid dynamics. Most liquids you encounter on a day-to-day basis follow Newtonian physics – the viscosity of the fluid tends to decrease as temperature increases. Melting butter in a pan, or heating up honey, are some common experiences that come to mind.
Cornstarch suspended in water, on the other hand, doesn’t follow Newton’s law of viscosity. It is a dilatant (or shear thickening) material.
Most people, I’ve found (and the studies show, as we’ll see), generally operate like this suspension. The more pressure you put on them, especially quickly (maybe, in response to some deadline), the more likely they are to freeze up like a tub of oobleck. They’ll, well, choke.
Why?
The Research
As you might imagine, a lot of research has been performed on cognitive ability and external forces, be that rewards or punishments (carrots or sticks). Sian Beilock, in “Choke: What the Secrets of the Brain Reveal About Getting It Right When You Have To” notes:
… counterintuitively, people who have the most ability to focus, the most working memory, and the most fluid intelligence, are more prone to perform poorly under stress.
Does that sound like anyone we talked about before? Maybe your high performer? Or, in general, high performing knowledge workers?
But maybe you don’t believe it. Perhaps you think that your brand of management hammer works best when brought down upon your reports.
Is this person, maybe, a member of a group under-represented in technology? Studies have shown that just being aware that women are “supposed to be bad at math” negatively impacts their performance (see Christine Shenouda Studies Impact of Gender Stereotype Threat on Girls’ Performance and Interest in Math).
When you belong to a certain group (say your gender is female, or you’re an African American, or any type of group), she says, “and that group is stereotyped as being not as good at whatever it is (such as gender in math), if you’re somehow reminded of that stereotype, then you’re going to perform worse on a test than if you’re not reminded of it.
Maybe – just maybe – they choked partially because you setup a system that hinders psychological safety? Why didn’t the lead tell you they weren’t finished? What feedback have you given people in the past when they came to you warning they were going to miss a deadline? Did you exhort them to work harder? Tell them to come in early and stay late? Or did you work with them to figure out how to cut scope? Did you tell them to (a phrase I’ve heard a lot) “Don’t come to me with problems, come to me with solutions”?
You may be a part of the problem. Yes, it’s their job to do the thing you’re paying them to do, but humans will behave in a way that fits within the system you’ve designed. And maybe you’ve designed a system that contributed to their choke.
Let’s talk about one of the two most important systems you design as a manager: the hiring system.
Hiring
What’s your hiring process? How do you interview people? Sketch it out. It might look something like this:
A candidate is interested in your job posting. They have a career of increasing performance (at least on paper) and have been lead developer at their current company for the last six years (prior to that, were a Sr. developer there for a few years, and had a couple of 2-3 year stints at startups in your area).
- Candidate talks to recruiter
- Recruiter passes resume to hiring manager (HM). You, maybe.
- HM reviews and has an initial screen.
- Candidate is brought to an onsite
- At the onsite, they rotate through various people, maybe every 30 or 60 minutes
- You have the candidate perform coding challenges, maybe on a whiteboard
- They meet with some VP or, in small companies, maybe the founder/CEO
- They answer various off-the-cuff questions posed by the team (you’re not quite sure what these are)
- You debrief, perhaps a few days later, with your team
- Someone says: “I 👎’d them because they couldn’t tell me what Big O of inorder traversal was of a binary tree.”
- Someone else says: “They couldn’t identify their most favorite feature that they’ve worked on. 👎”
- The VP says: “When I asked them if programming was their passion, they took too long to say ‘Yes’. 👎”
Did you make a good no hire decision? Why?
Would you come to the same, or different, conclusion if the candidate has been out of work for 3 months because their startup went under? What if this person was a member of a group under-represented in technology?
What if your candidate is an otherwise high performer who just choked because of your interview process? Would you be able to detect that? Let’s explore.
Moar Research
In general, the link between pressure and performance has received much academic attention over the years.
… Dandy, Brewer and Tottman (2001) showed that Australian basketball players’ free-throw performance was worse during games than during training. In addition, Dohmen (2008) found that male professional soccer players’ performance is negatively affected by the presence of a supportive audience. More recently, Hickman and Metz (2015) showed that higher stakes increase the likelihood to miss a shot on the final hole in professional golf.
Lots of studies have shown that the larger the financial reward, the worse the performance of someone performing creative or cognitive tasks (see: “Large Stakes and Big Mistakes” and of course, Daniel Pink’s “Drive”).
What could be more financially rewarding than a raise at a new position? Or worse, when someone is unemployed, the thought of running out of savings vs. a new six-figure job?
How Can We Fix This?
You may think this isn’t a problem. Maybe the candidate that chokes in the interview is going to fail at your company. Possibly. Although the evidence doesn’t suggest that. If you’re having trouble hiring or retaining good people, it’s worth figuring out how to build a better hiring process and how to increase psychological safety at your company.
The Hiring System
What sort of systems are we optimizing for when we hire a new person? What are we testing when we hire? The ability to interview? The ability to derive Big-O of some algorithms? Is that likely to be how they perform at work? Manager-as-systems-builder comes into play once again in the hiring process.
Do you have a standard list of questions you ask? Do all the people who are interviewing them have this same list? Do you know what’s on it? Do you review this list periodically?
Interviewing is a skill, just like anything else, and it’s not likely that everyone who is interviewing a candidate is great at it. Some might be better at interviewing for cultural add, some might be better at highly technical things. Given you have a short amount of time with a candidate, the last thing you want is three people asking the same question to the candidate. Or, having people forget to ask a critical question and you need to reach back out to the candidate for clarification. Your job, as a manager, is to build a hiring system, not a process.
Consider Topgrading and the STAR method. These techniques can be boiled down into starting with a pre-defined list of questions that focus on key job areas/skills/experience and digging in until you are fully satisfied the candidate has demonstrable expertise in that area. Whenever the candidate says “we” (which might happen the higher a person gets in their org chart), re-direct to get to “I did” statements.
Your list of questions should cover (if applicable) important technical topics and cultural elements. If you are a JavaScript shop and expertise is necessary, then make sure you ask sufficient questions to demonstrate this expertise. If one of your cultural elements is “kindness”, what sorts of questions can you ask to have the candidate exhibit their kindness?
Aside: Given the link between higher-skilled people and choking, what does that imply about lower skilled people? Would they choke less under pressure (or just not be able to do it at all)? What implications does that have on how you evaluate people in the interview process? Perhaps the lower skilled person can recite Fibonacci algorithms – but be unsuited to working at your company – but the higher skilled person can’t? Food for thought.
Note that you don’t need to personally cover all the topics, but your entire interview panel should have complete, slightly overlapping, coverage. Sit down with your team and discuss the rubric. Make sure that more than one person has the same topic (sometimes people might not get to that question). Have each team member write out their list of questions, and discuss with the team. Is that question trivia, or is it diagnostic and relevant to the kind of work you do at your company? What potential biases are you introducing with this question? Consider having different questions tailored to the person you’re interviewing.
Make sure everyone does not deviate from this list of questions. We’ll talk more about this later, but the goal is to create a repeatable, scalable hiring system and improve it over time. If people are asking random questions without any consistency, it will be a challenge to gather any actionable data as to why a hire worked out (or didn’t).
An Example Interview
You may have a list of questions (for a Eng Manager) that might look like this:
- Tell me about a time when you had to fire someone
- Tell me about a time when you disagreed with your boss
- Tell me about a time when your team missed a deadline
Ask the candidate those questions, but then pivot the conversation based off of their responses. A sample dialog might go something like:
HM: Tell me about a time you had to fire someone.
Candidate: We had this engineer that kept showing up late and they smelled bad, so we decided to let them go.
HM: Why were they consistently late?
Candidate: Had to take care of a dying spouse.
HM (Wow! This might go against ‘Kindness’)
… More pivots based off of candidate answers …
HM: Tell me about a time your team was late delivering something.
…
etc.
For an individual contributor, consider having some prep-work (a take-home assignment) that you can continually work on throughout the interview. One I did (long ago when I was hiring Ruby devs) was:
- Create a Ruby class to model a deck of cards (we’ll want to play solitaire with it; link to Wikipedia for Cards, Solitaire, and a free online Solitaire game).
- Consider the different operations you might want to perform on this (deal, shuffle, etc.). Do those need to be in this class or somewhere else?
- Do you need tests for this? What tests might you create?
This is super simple (you can likely scale this example from Jr to Sr levels). They can read about cards (in case they haven’t been culturally exposed to solitaire or cards) and can write a quick, naive class, in a few minutes. Then, in a phone interview, ask them to add certain methods to it. How would you sort the deck? How would you write the test to verify that the deck was sorted?
In the onsite, you could (if you were a web shop) say: let’s take this deck class and build a web GUI around it so that people can play solitaire.
In past lives, we had an open-source trivial app (say, “Where should we eat lunch today?”) and we’d pair with the candidate and add some feature (we’d ask them to come prepared, if possible, to add some new things to this app). That type of work was representative of how we did things at our company (pairing was an important part of our process). We’d throw away the feature so that it could be re-used by other candidates.
Now the Controversial Part
Share this entire process and list of questions with the candidate ahead of time. This may surprise you. You may object “That gives the candidate time to lie about their answer!” Maybe. But that’s the goal of the topgrading/STAR process – your job is to take their initial answer and dig in to expose any deception or places where the candidate glosses over their experience.
Asking someone, unprepared, a list of questions will increase the likelihood that the candidate chokes and you get a false negative. The more you can share with a candidate ahead of time, the less risk of choking. And, it’s unlikely they will need to do such things in their job on a routine basis (unless, of course, you are writing binary search trees and you need a binary search tree expert, having them know what Big-O of some algorithm is).
Aside: Do you think the FBI is an organization that has some amount of secrecy and high requirements for honesty? Yes, they have a polygraph, but their entire interview process, including a list of sample questions for each stage, is all available online. Do you have data/evidence to suggest your process requires more secrecy than the FBI? If so, maybe you want to do something else, but chances are, you’re turning away good candidates.
After Interview Retrospective
In the meeting that should happen immediately after the candidate interviews (leave some time for this at the end of the day!) you should discuss the questions that were asked, and whether or not people Hire or No Hire.
If an interviewer No Hires because they didn’t have sufficient evidence, then see if someone else gathered that. Sometimes you’ll see a No Hire switch to a Hire after this discussion, and that’s OK as long as you agree the candidate demonstrated that particular experience.
If the candidate is someone who is from a group under-represented in technology, they are more likely to choke in an interview. If you’re using the same rubric for white men as you are for people in those groups, you’ve got a biased rubric.
Long-term Retrospective
Review this system periodically. As your company changes, your rubric and questions may need to change. When you need to let someone go, review these questions and see if there was a gap in your coverage. What did you miss when you hired them?
Performance Management
Let’s circle back on your top performer’s issue from before. Knowing what we know now, what is your take on it? What might you consider doing differently?
Optimal people performance, as we’ve discussed and will show further, comes from a system that: * Is designed to keep people operating automatically (in flow, common/automated practice, etc.) * Has high organizational/individual safety * And employs other techniques to reduce likelihood of choking
Automatic to the People
The studies have shown that, generally, folks choke less under pressure if they can perform in an automatic manner. Somewhat paradoxically, focusing too much on the details can cause folks to stumble. Relatedly, it seems the difference between a run-of-the-mill high performer (which are, admittedly, rare) and a truly exceptional person (even more rare!) is they choke less.
DUBNER: So let me ask you this, Anders: To what degree do you believe that choking is the factor, or a main factor, that actually separates an absolute top-tier performer from someone who’s talented but doesn’t reach the top tier? In other words, is the expert — is the professional — the very good performer who has learned to not choke?
ERICSSON: My experience is that choking is quite rare by those individuals that we study, who are consistently excelling. And it seems to be almost part of being an expert is that you deal with the kind of situations that would be experienced as very high-pressure for other people.
This has a couple of implications:
- As a manager, you should limit the distractions that kick people out of flow state. The process of operating autonomically requires moving past the context setting and into flow.
- You may consider moving folks into private offices if they are in shared plans and on a critical process (better: let people work from home!)
- Change very little process during these highly critical times (don’t add standups if you don’t have them, don’t have more meetings, etc.).
- Build a culture of shipping early and often. If shipping software is something you don’t do very often, by definition it’s a unique process and causes higher pressure.
Organizational Safety
There’s a link between high performing orgs and high psychological safety. I encourage you and your team to take this test and find ways to increase your team’s safety. That is likely a series of courses in of itself, but some useful articles:
- High-Performing Teams Need Psychological Safety. Here’s How to Create It
- How to Create a Culture of Psychological Safety
Turning High Performers into Experts
If the difference between an expert and a high-performer is that they choke less, can we turn high-performers into low-choking experts by experimenting with ways to get more muscle memory in high-pressure experiences?
Firefighters train in actual (staged) burning buildings. Sports teams often blare loud music and crowd noise during practice. They practice as they perform. It seems unlikely that we’ll want to put extreme pressure on our folk, so the alternative seems to suggest that we ought to reduce the high-pressure circumstances by repeating our normal routine as frequently as possible.
Continuous delivery and delivering in small batches has been linked to high performing organizations. If you can fail small, and earlier, it can build your confidence and reduce choking in the future.
Alternately, we should work on reducing stress and high-pressure situations:
BEILOCK: Yeah, and we’ve actually shown that getting people to just jot down their thoughts and worries can be beneficial, just sort of downloading them from mind when they feel stressed out. And one of the things that that does is get you to realize maybe it’s not such a big deal, right? What you’re doing is not as big of a deal as your friend getting hit with the club and dying.
DUBNER: Again, always comes back to dying. What about other means of directing the mind, whether meditation, perhaps?
BEILOCK: There’s lots of research showing that meditative practices can help change how you focus, and your ability to focus on what you want, and get rid of what you don’t. That’s true with visualizing positive performance outcomes ahead of time, and really focusing on why you should succeed. What are the factors that you’ve practiced well? You’ve got this. You’ve had situations like this in the past and they’ve gone really well.
Further reading
Epilogue
If you want to better hire folks, and have higher performing teams, you should:
- Understand how and why people choke in stressful situations, and develop mechanisms to help your folk avoid choking
- Come up with a defined hiring process that focuses on relevant, demonstrable skills they’ll use in the job
- Train your interviewers on how to interview, review their questions, conduct mock interviews
- Periodically review the hiring rubrics for relevancy, missing/duplicate content, and bias
- Make high organizational/personal safety a goal of every manager
- Develop both technical and people processes to reduce risk
- Create systems the promote automatic behavior (CI/CD, Kanban/flow state, etc.)
- Find ways for your folk to practice like they play
Special thanks to the following reviewers:
- Caleb Huitt
- Josh Kaderlan
- Alden Peterson
- Steve Ruben
If you want to see an epic choke, watch this video:
