Science series materials are brought to you by TestGorilla’s Assessment Team: A group of IO psychology, data, and psychometric experts with a deep understanding of the science behind skills-based hiring.
If you read our previous blog post about selection error, you’ll remember that we defined exactly what it is and explained that you can make better hiring decisions by reducing it.
Here, we’re taking it one step further. In this post, we’ll expand on those points by showing you how using valid, reliable selection tests during your hiring process can reduce selection errors, ultimately leading to more successful hires and fewer mis-hires.
Before we get started on the role that valid, reliable selection tests can play in minimizing selection error, here’s a quick reminder of what selection error, selection hits, and selection misses actually mean.
Selection error, in the context of employment screening and hiring, refers to mistakes made during the selection process that lead to the hiring of unsuitable candidates or the rejection of suitable candidates (i.e. selection misses).
A selection miss occurs either when an employer hires an unsuitable candidate (this is known as a ‘false positive’) or rejects a suitable candidate (this is known as a ‘false negative’).
A selection hit occurs either when an employer hires a suitable candidate (this is known as a ‘true positive’) or rejects an unsuitable candidate (this is known as a ‘true negative’).
You can see a breakdown of the different types of selection hits and misses in the table below. In summary, selection error must be reduced if employers want to maximize hits and minimize misses during hiring: minimized selection error = better hiring decisions.
As the definitions above indicate, the goal of any hiring process is, ultimately, to land selection hits – to hire suitable candidates and reject unsuitable ones. Pre-employment tests come in handy here: By using valid and reliable selection tests, employers can actually increase their selection hits.
Before we explain exactly how and why they can do this, we’ll explore exactly what we mean by the words “valid” and “reliable” in the context of selection tests. That way, you’ll understand why tests need to be valid and reliable in order to successfully minimize selection error during the hiring process.
Test reliability refers to the degree to which a selection test works consistently. In employee selection, this consistency takes various forms. For example, a test must measure a skill similarly, regardless of who is taking the test.
Suppose you bought a new phone to make important business calls. Your new phone has valuable features and high sound quality. However, it randomly powers off while making calls about 15% of the time. While you may like the phone’s features, you cannot rely on it for business calls due to inconsistent performance. You need a reliable phone to make business calls. Similarly, you need reliable selection tests to make hiring decisions.
An employee selection test, such as a skills-based hiring test, must work each time you use it. The degree to which a test works consistently refers to its reliability. You can find more information about reliability in our guide to psychometric properties of pre-employment tests.
Test validity refers to the degree to which a selection test measures what it’s intended to measure and is associated with test accuracy. For example, a skills-based test should include test items that fully assess the target skill area and avoid assessing irrelevant characteristics of the candidate.
In short, valid selection tests consistently assess what they were designed to measure. For more information about test validity and its different forms, read our brief introduction to test validation.
Valid, reliable selection tests can increase selection hits by improving the accuracy and effectiveness of the selection process. When organizations use them, they are more likely to hire candidates who are a good fit for the job and the organization. Outlined below are three ways that valid, reliable selection tests reduce selection error:
Valid, reliable selection tests assess candidates objectively based on job-related criteria. Alignment between the selection test and the job role ensures that candidates who perform well on the selection test possess the necessary qualifications to do well on the job, increasing selection hits.
Valid, reliable selection tests focus on assessing job-related candidate characteristics and avoid assessing personal characteristics irrelevant to the job. Removing bias and subjectivity, valid, reliable selection tests are more objective, leading to better (and fairer) hiring decisions.
Valid, reliable selection tests predict future job performance accurately. When candidates perform well on these tests, there is a higher likelihood that they will excel in the actual job role because they’ve been evaluated on job-related criteria.
A manufacturing company called Space Electrics is hiring machine operators. As part of the hiring process, they administer a Machine Operator test that assesses candidates for skills using a certain machine – let’s call it Machine A1.
The selection test is outdated, because employees at the company now use a different machine – Machine A2. The Machine Operator test includes test questions that focus on the now irrelevant processes of Machine A1, and fails to assess important operating processes of Machine A2. This means it’s not valid. Because the test is outdated, it also uses language that isn’t job-relevant, and that a subset of the candidate pool would be unfamiliar with. Since it has a systemic bias towards a subset of candidates, it is also unreliable.
Space Electrics used the invalid, unreliable selection test to assess 140 candidates. They saw the following results:
30 false positives: Thirty candidates who know how to operate Machine A1 but not Machine A2 were hired and performed poorly
45 false negatives: Forty-five candidates who know how to operate Machine A2 were not hired where they could have performed well
20 true positives and 45 true negatives, with 40% of those hired being successful hires (or true positive hits)
False positives and false negatives increased Space Electrics’ selection misses (75 candidates) and decreased its selection hits (65 candidates).
Space Electrics is hiring a second cohort of machine operators and has since become aware of the Machine Operator test’s issues. As part of the hiring process, they will now administer the Revised Machine Operator test that measures skills in using the more up-to-date machine – Machine A2 – and uses job-relevant language.
Space Electrics used the valid, reliable selection test to assess 140 candidates. They saw the following results:
10 false positives: Ten candidates who know how to operate Machine AI but not Machine A2 were hired and performed poorly
15 false negatives: Fifteen candidates who know how to operate Machine A2 were not hired when they could have performed well
40 true positives and 75 true negatives, with 80% of those hired being successful hires (or true positive hits)
False positives and false negatives decreased from the first hiring phase, so Space Electrics’ selection misses decreased (25 candidates), and selection hits increased (115 candidates).
In the examples above, using valid, reliable selection tests ultimately led to better hiring decisions, increasing the percentage of those hired who were successful hires (or true positive hits) from 40% to 80%.
The business impact of such a change is hard to overestimate: Doubling the rate of successful hires means reducing turnover, minimizing training costs, and it ensures fewer future hiring rounds are necessary.
For employers, ensuring selection tests are valid and reliable leads to the following key benefits:
Reduced bias and subjectivity: Valid, reliable selection tests focus on assessing job-related candidate characteristics and avoid assessing personal characteristics irrelevant to the job. Removing bias and subjectivity, valid, reliable selection tests are more objective, leading to better hiring decisions.
Increased diversity: Valid, reliable selection tests reduce bias and discrimination in the hiring process, helping your organization to identify and hire diverse talent who will perform well and add to your organizational culture.
Improved culture fit: Valid, reliable culture add tests can lead to greater culture fit as you maximize your likelihood of selecting candidates who will add to and align with your organization’s values.
Reduced turnover rates: Turnover rates decrease because employees typically stay in roles that suit them. Hiring the right candidate from the start reduces the costs of recruiting, training, and onboarding new employees due to hiring misses.
Improved legal defensibility: Valid, reliable tests help organizations ensure their hiring practices are legally defensible by reducing the risk of discrimination and legal challenges in the hiring process.
Without monitoring validity and reliability, employers could miss out on these benefits. That’s why it’s important to a) get your tests from a provider who continuously monitors and improves them, and b), monitor their reliability and validity for the roles you’re using them to select candidates for.
In summary, valid, reliable selection tests increase selection hits by ensuring you evaluate candidates based on job-relevant criteria, reducing bias and subjectivity, and predicting future job performance accurately. They ultimately lead to better hiring decisions, increased selection hits, and improved organizational outcomes.
The benefits of using valid and reliable selections are manifold, and TestGorilla’s Assessment team works hard to ensure we create valid and reliable tests. We also conduct continuous monitoring and improvement of existing tests in our test library – check them out! Some of our newest tests include the Microsoft Project test, the Video Content Creator test, and the Pharmacy Technician test.
If you want to learn more about the science behind TestGorilla, we recommend reading:
Create pre-employment assessments in minutes to screen candidates, save time, and hire the best talent.
No spam. Unsubscribe at any time.
Our screening tests identify the best candidates and make your hiring decisions faster, easier, and bias-free.
This checklist covers key features you should look for when choosing a skills testing platform
This resource will help you develop an onboarding checklist for new hires.
How to assess your candidates' attention to detail.
Learn how to get human resources certified through HRCI or SHRM.
Learn how you can improve the level of talent at your company.
Learn how CapitalT reduced hiring bias with online skills assessments.
Learn how to make the resume process more efficient and more effective.
Improve your hiring strategy with these 7 critical recruitment metrics.
Learn how Sukhi decreased time spent reviewing resumes by 83%!
Hire more efficiently with these hacks that 99% of recruiters aren't using.
Make a business case for diversity and inclusion initiatives with this data.