In the 1976 Summer Olympics in Montreal, an 86-pound, 14-year-old Romanian dismounted the bar after a breathtaking performance. As the wild applause calmed down, everyone stared at the judges. The scores were finally flashed on the screen and the crowd, and especially little Nadia Comaneci, was bewildered — a seemingly low rating of only 1.0! It was a simple problem — no one had ever anticipated a perfect score in Olympic gymnastics and the equipment was not designed to display three digits. Nadia’s perfect performance was not a fluke. She went on to achieve another five perfect 10 scores in her career. How did the judges know and agree that six times Nadia’s performance was deserving of a 10.0 and not a 9.9?
Robert Parker reviews wine on a 100-point scale (it is actually only a 50-point scale: he never rates a wine below 50 points) and is considered the world’s biggest wine critic, not because he is a grouchy old man, which some would say he has become these days, but because more people in the world are influenced in their wines-choosing behaviour by him than anything else. Most critics, such as a movie or restaurant reviewers, have influence in a very local, and thereby limited market. Parker’s influence is very apparent wherever wine is sold and is thus international. He is well aware that the difference between a rating of 90 and an 89 is critical, not just for the retail store, but for the winery. At a 90 rating, wines fly off the shelves. One point less and they linger. How does Parker know when to rate a wine a 90 versus an 89?
Comparisons and ratings happen all around us. One can look up a rating on almost anything: Consumer Reports, ‘Rate Your Doctor’ on the Internet, and mutual funds ranking for investment purposes. Judging, rating, and comparing is something humans do naturally, almost unconsciously. It should be no surprise that organizational life specialists in human resources have developed numerous ways of rating candidates and jobs. There are behavioural rating scales for interviews, compensation studies on positions, performance ratings, 360°-feedback evaluations, and the list goes on. But why hire this candidate versus another?
Into the field of HR practices enters the industrial/organizational (IO) psychologist, a person trained in the science of individual differences who can declare a particular candidate is a 4 on a 10-point scale and should not be hired. Should the hiring manager take heed? Like any rating, it all depends on the confidence one has in the rater.
When Olympic judges award a perfect score, some in the audience may differ in their views. However, most of the judges are former gymnasts themselves and have spent years of judging the sport to rigorous peer standards before qualifying for being an Olympic judge. Their scores are supposed to be unbiased and impartial as they rate a performer on difficulty of routine, the execution of somersaults, flips, and balance. There is a system for judging a gymnast, as there is for judging ice skaters, divers, etc., and judges are deemed highly skilled at applying it. However, subjectivity does arguably seem to creep in, as judges seem to have a bias towards rewarding higher points to performers of their own country. One can easily see the importance of having more than one judge in such situations.
When Parker judges a wine, it is obviously his subjective experience of the wine. However, he follows a fairly exact protocol which he defines and describes clearly for his followers. He looks at its colour and clarity, smells its aroma, notes how it clings to the glass (higher alcohol wines are more viscous) to the glass, and then rates the wine on a number of factors known as structure and age worthiness. He can articulate why one wine should be rated 90 points and another 89 points and is well aware, often based on the flood of criticism he receives from winery owners, what will happen when he rates a wine as lower. The world is flooded with wine, and when Parker gives an 89 rating he is telling us that there are likely better wines for your dollar.
To the IO psychologist, Parker has an interesting talent. He is able to predict the ‘potential’ of a wine. He made his reputation on predicting that the 1982 Bordeaux vintage would be one of the best of the century. Almost every wine critic at that time had panned the 1982 vintage, saying it was too hot of a year and the wines would never evolve or last long. They were wrong, very wrong, and Parker was exceptionally prescient in his evaluation of the vintage. It launched his career from mediocre lawyer to the world’s biggest wine critic. The ability to accurately evaluate potential can be worth a literal fortune in any industry. Knowing the potential of a candidate is of much value to a company.
The science of evaluating candidates involves the experience of Olympic gymnasts judges and the skills of Parker’s ability to determine ‘potential,’ and much more. However, the way in which IO psychologists judge candidates is very different from the way Olympic judges, wine experts, or managers judge candidates. IO psychologists essentially use science and advanced statistics to be as impartial and non-subjective as possible. The instruments they use are developing and validated on a specific population. This can involve administering a battery of tests to all the members of a defined working population and then correlating test scores with actual job performance. A test is not valid unless it can be clearly demonstrated that the lowest performers generally receive the lowest scores on the tests, and the highest performers the highest. Based on the validity study data, the average score for that population can be easily calculated and cutoffs for hiring new employees determined. A good instrument, by definition, should clearly show the benefit of hiring candidates above a certain minimum standard.
IO psychologists, depending on their specialty, also are usually skilled in personality assessments. Personality is very complex and can include hundreds of ways to describe people. Take any of the 4,000 adjectives in the English language, and you can describe any human characteristic or behaviour with increasing degrees of subtlety. It is easy for most people to see that a candidate is outgoing and extroverted in behaviour, or shy and introverted, but it is another level of analysis to say that the candidate has the potential for emotionally exploding under pressure when he/she is sitting in front of you as calm and composed as a saint. It is highly unlikely that the candidate’s explosive personality will ever be demonstrated in an interview – candidates are, of course, on their best behaviour, and their relating of their past and how they do things may be quite different from how others would describe it. Less than ideal examples of their own behaviour are often glossed over, or actually forgotten. Even if confronted with glaringly bad references, the candidate will often explain the unfairness of the situations they found themselves in – an authoritarian boss, a hopeless company, hapless colleagues – and often paint a picture of heroism in terms of how and what they did to overcome such adversity. The candidate is always a biased reviewer of his or her own behaviour and performance. Oddly, sometimes this bias is negative in the sense that candidates see themselves as ‘average’ or lacking in certain abilities, whereas the interview, their references, and the test results tell a very positive story.
When an IO psychologist judges personality, some subjectivity will be involved. They need to apply the same rules to themselves and know that it is human nature to be subjective. They may have a preference for this or that type of personality over another. However, there is a critical difference between how a psychologist approaches evaluating a candidate versus how a manager might. When a manager interviews a candidate and says, “I like this candidate. I think this candidate can do the job,” how did he/she arrive at this opinion? The manager may like a candidate’s answers to certain questions, how the candidate performed in the interview, or simply how the candidate looked. There is much research showing that biases creep into interviews at every opportunity. Hearing negative information can have three times as much weight as positive information, and many untrained interviewers make up their minds about a candidate mere minutes into the interview and spend the rest of the time looking for information to confirm their view. This, of course, happens unconsciously, and without any feedback about the accuracy of those perceptions, it will continue to happen regardless of how many times the manager conducts an interview.
Trained IO psychologist are aware of the human nature of biases, and it is one of the primary reasons why they develop and use tests and base their predictions on the tests, not their own perceptions. Tests are indifferent to candidates; they neither like nor dislike them. They are emotionless and objective, i.e., unbiased raters of a candidate’s ability and potential. However, psychologists and instruments are not perfect so what are the risks for not heeding their recommendations? There are a number of things to consider:
1. The odds are against you
If you hire a candidate who was rated a 4 or lower on a Stefan, Fraser & Associates personality testing, the odds are significantly greater that the person will not perform to the level of the average worker in your organization. In most situations with our clients the tests were validated on the employees in your organization and regardless of what you feel or believe you see in a candidate based on interviews and reference checks, in the long-term the statistics will be against you. In the past candidates performance on the tests in your organization were directly correlated with job performance.
2. Your reputation is on the line
As a manager, your reputation is based on many factors, mostly the ability to do your job. However, most managers achieve their results via others, i.e., subordinates. When you hire candidates who rate a 4 or lower, you will be less likely to achieve results through them. Low-rated candidates will likely fail to perform the way you want and you will likely spend significantly more time with them on performance issues than higher-rated candidates. Your reputation will also suffer if others view you as reluctant to use the tools and knowledge that the company has paid for in order to help you select better candidates.
3. Often unpleasant surprises exist in candidates rated 1-4
There are problems on the so-called ‘left side’ of the bell curve. As cognitive ability declines, the darker side of human nature emerges. Many criminals in our penal system are not bright people. Almost by definition, they have demonstrated their lack of intellect – they got caught. Approximately half of the 1-to-10 rating scale is based on cognitive abilities. We unfortunately find that the lower the score, the higher the number of incidents of negative work behaviour. In the testing industry, this is called counter-productive behaviour and ranges from insubordination, dishonesty, lack of integrity, fighting with others, increased accidents, and theft. These behaviours take many hours of management and HR personnel’s time to address. Investigation, discussion with union representatives, discipline, and firing a candidate all detract from the real purpose of managers and the company, lower productivity, and add to hours worked by hassled managers. By hiring high-rated candidates, the manager will avoid the endless future hours of problems that typically come with low-rated candidates.
4. Your decision is a company decision and affects all
When you hire someone for an entry-level position in your organization, especially a young candidate, you are making approximately a million dollar decision on behalf of the company. The company will need to see a significant return on this investment, and if the candidate is mediocre or weak, you have lowered the competitive advantage of the company in the global marketplace. As a manager, your goal should be to hire the good talent and refuse to accept candidates that are below your average performing employee in the company.
5. Best deserving candidate
When you hire a candidate you are often awarding that person with a significant opportunity of a lifetime. This person will have a career, and a high paying one at that. Is this candidate deserving of this award? When you fill a vacant position, the position may be filled for the next 35 years. Is this individual the most deserving candidate in the marketplace? If you hire a mediocre candidate, you have denied the opportunity to a better candidate. When the company awards positions carefully, they will be rewarded many times over for doing so.
In summary, in the Hiring Olympics there are those who will be medal winners and those who will be at the end of the pack. Having well thought-out and scientifically validated instruments and processes to sort out the contenders before the race starts is a powerful HR tool that is available for every manager to use.
