The wrong hiring decision can be expensive…very expensive. That’s not just in theory. And while we oftentimes conduct a post-mortem analysis once the mistake is realized, we generally focus on the shortcomings of the candidate versus the shortcomings of our decision process. To illustrate, I’ll share a story that could easily have cost my client hundreds of millions of dollars.
I was retained to conduct a search for a successful software company. The search process was set up to be as streamlined as possible, with a point person managing the search internally and taking the first pass at interviewing candidates. Then, a small and select group of second interviewers, peer executives and board directors, were slated to spend time with the one or two finalist candidates.
The point person was a seasoned and senior executive who had hired many people over their career. The select group of peer executives and board directors were also experienced interviewers, team builders, and had assisted in the hiring of many key execs throughout their portfolio companies.
We came down to two finalists. It was clear to me that one candidate, Peter, was the better choice. However, the client interviewing team expressed a preference for Carl, the other candidate, who was referred into the process through a board director.
Carl and Peter had both gone through the same rigorous interview and assessment process that I use with all executive candidates. From a professional fee perspective, I was covered regardless of the source of the referral. As always, my commitment was to make certain that my client made the best hiring decision regardless of whether we had originally identified and introduced the candidate.
However, the situation demanded that questions were asked, both of myself and of my client:
- Was I unintentionally favoring Peter because he was “our” candidate or was my client favoring Carl because he had been referred into the process by a board director?
- Was my client not seeing what I was seeing, or, was I missing something in my evaluation of the two candidates?
- Importantly, had some unintended bias influenced either the client interviewing team or me leading to a cognitive error in judgement?
It is important to note that a bias can concern anything, and it can be exercised with individuals who are the most educated, experienced, and are at the highest levels within their organizations.
To explore what could be at work in this circumstance, we need to look at the multiple types of evaluation errors common in any assessment process:
- Errors of Anchoring (fixating on a single data point)
- Availability (applying a past conclusion to a present similar situation)
- Attribution (reaching an overarching conclusion based on a single observation)
- Confirmation Bias (ignoring or rationalizing contradictory data to fit a conclusion)
- Satisfaction of Search (looking for a single cause or reason when the source of the issue is multiple reasons), and
- Prototype or Representativeness Error (applying a stereotype based on an observed similar characteristic.)
While all these errors can be distinguished as separate cognitive phenomenon, they all have biased thinking at their core. In the context of evaluation and assessment, all lead to the result of ignoring data as it is presented, or stopping the investigative process to discover additional data, or influencing the investigative process itself so that discovered data fits into an already reached conclusion.
If we take a simple fact about someone that is easily discernible from reviewing their resume – a Harvard education, for instance - pre-determined thoughts on the meaning of a Harvard education can become biased thinking which can then lead to cognitive errors of evaluation.
For example, it would be common to assume that someone with a degree from Harvard is smart, but it may be that the candidate was accepted into and graduated from Harvard principally leveraging their family’s wealth and/or connections. In an anchoring evaluation error, an interviewer would cling to the assumptive conclusion about the Harvard grad’s intellect, despite potentially being presented with contrary evidence.
In an availability error, an interviewer who might have had numerous positive experiences of individuals with degrees from Harvard, who exhibited additional characteristics such as drive, ambition, common-sense knowledge, fairness and discernment, would presumptively conclude the candidate also had these characteristics. Then, the interviewer would not pursue additional lines of questioning to determine whether this conclusion is true.
With an attribution error, the interviewer would apply a stereotype they’ve developed, whether positive or negative, around the fact that a person graduated from Harvard. Then, regardless of evidence that may be presented in the interview with the candidate, the interviewer perceives the candidate as representative of that stereotype.
These errors can all lead to confirmation bias in the interview process, shaping the line of questioning, ignoring contrary data and/or force fitting data into a continuously self-reinforced conclusion. And then, post the interview, the path towards cognitive error continues in the analysis of the discovered data.
As stated earlier, all executive candidates are put through a similar rigorous interview process. I will spend three to four hours with CEO candidates and two to three hours with candidates who are direct reports to the CEO. These are highly structured interviews, with carefully thought-through questions, designed to allow candidates to tell stories about their careers and their lives - stories which elicit details around what is important to them, as well as when they have been in circumstances most like the ones they would confront if they became the hired candidate. As a candidate tells their stories, questions are asked to drive greater clarity around details. By actively listening, an understanding is developed at both a detailed and holistic level about what they did, how they did it and what thinking drove their actions leading to their results.
During these interviews, not only are voluminous notes taken, but the interviews are also recorded and then transcribed. This allows reliance on more than memory and notes, or the overall experience of the interview, ensuring important data is not missed. The interview can then be re-listened to, or in the case of a videoconference, re-viewed to discern the candidate’s expression, posture, energy and passion infused into certain anecdotes, or the lack thereof.
All this data is fodder for the analysis and generation of a detailed interview report that describes what a candidate has done and how they have done it as well as their strengths and possible weaknesses.
After distilling to several potential finalists - generally two to three candidates - I also use a tool, developed and refined over many years, based on the Kepner-Tregoe Decision Analysis Methodology. This is an analytical methodology and structured approach for gathering, prioritizing, weighing and evaluating information on competing alternatives. Using this tool brings more objective standards into the decision process, forcing careful thought across multiple and complex criteria.
The overall process described thus far can be simplified to:
- Structure the interview.
- Create specific questions that allow a candidate to tell more complex stories and deliver deeper data about the circumstances they have been in, their thought process to devising solutions within those circumstances, actions taken, the results produced, and lessons learned.
- Put in place a mechanism beyond your memory and notes to collect as much and as accurate data as possible.
- Practice Active Listening.
- Analyze the total collected data and write an assessment.
- Create an analytical decision tool that allows you to re-examine your conclusions and compare alternatives while prioritizing, weighing and ranking options across multiple categories.
These steps, regardless of what specific tools used, force attention to be paid to your biases, lessening their impact in the evaluation process and reducing the probability of making unintentional cognitive errors.
So, back to the story of what happened in this search…it was clear to me that Peter was the superior choice over Carl, but how did I “know” Peter was the better candidate, and what led me to that conclusion?
When my client expressed a preference for Carl, I immediately constructed a comparative evaluation matrix, re-reviewed and re-analyzed the data while completing the decision matrix by assigning ranks for each candidate across the multiple criteria distinguished. Each candidate’s ranking was based on direct evidence, the stories and exact words used and the way in which each person communicated. Once completed, I even readjusted the weighting and scoring to potentially compensate for any natural bias I had for Peter over Carl.
The results were clear. Peter was the better candidate and fit.
I shared these results with my client, walking them through the decision matrix, my analysis and the data used to validate my rankings of Peter and Carl across the multiple categories. This forced my client to re-examine their conclusion where they saw that they had not considered the full breadth of data. There were, in fact, hidden biases at work and cognitive errors in judgment preventing my client from reaching a conclusion based on an objective as possible analysis from a clear view of the facts.
The reference-checking process further validated the accuracy of Peter’s assessment and scoring on the decision matrix. He was hired by my client and the company ended up with a phenomenal exit, largely driven by Peter’s contribution and the difference he made overall to the company’s growth and development.
Would Carl have made as big a difference as Peter? Both individuals were superior candidates, but my analysis showed that Peter had the higher probability of being the difference maker, the candidate who could deliver $500 million or more in value.
However, what I do know is the uniform feedback we receive from both clients and candidates is, we deliver the most thorough and intellectually rigorous process they have experienced. This process ensures that both candidates and clients are making the right choices, ensuring that hidden biases are not leading to costly mistakes.
 Some names and identifying details have been changed to protect the privacy of individuals and organizations; however, the magnitude of the potential cost was real and, in fact, understated.