Goodbye keyword games: AI resume scores have arrived, and experts are here for it

Let’s recap: The problem with traditional AI resume screening 

Traditional AI resume screening is based on something called “keyword matching.”  Here’s how it works: 

• A recruiter uploads a job description to software such as an applicant tracking system (ATS) or a stand-alone resume screening tool. 

• The tool reads the job description and extracts keywords relevant for the role – these could be skills, technologies, job titles, etc – for instance, “CRM management,” “HubSpot, or “senior leader.” 

• Then it scans each resume for these keywords. Those that have enough matching keywords (e.g., 70%, or as set by the administrator) are shortlisted, while the others are filtered out immediately. 

Over the years, we’ve seen some real problems emerge with this system. Let’s call these the “A-B-C problems of keyword-based models.”

A-B-C problems of keyword-based models

A – Accuracy issues 

Keyword-based screening tools look for words, but can’t read between the lines. For example, in the search for “project manager,” the AI system might reject a candidate who “led cross-functional delivery teams using Agile methodology,” but didn’t have the exact keywords. 

Additionally, AI resume screeners don’t understand nuance and context, which Jim Hickey, president and managing partner at Perpetual Talent Solutions, says is “a big issue in hiring.” 

For instance, these tools might shortlist all candidates who have “contractor” (meaning freelancer or consultant) on their resumes, when in reality, the word “contractor” in the job description was asking for construction contractors. These minor details are overlooked when AI resume screeners make selections with no human oversight.  

Hoegener explains the consequences: “You can end up filtering out great candidates who just didn’t use the right keywords. Or you spend time interviewing people who look great based on their resume, only to find out they actually don’t have the experience you’re looking for.”

Moreover, Hoegener also observes that “candidates are now using AI to craft ‘perfect’ resumes filled with the right keywords, and recruiters are using AI to scan them [...] Some even list skills they’ve never used just because the AI told them to [...] Hitting all the keywords doesn’t automatically mean they are a fit for the role or a top candidate. That’s gaming the system.”

B – Biases

Contrary to early hopes that AI would remove bias, “AI is only as fair as the people who build it and the data it’s trained on,” Hickey explains. “[This means] it remains just as preferential as humans. Worse, it can very quickly replicate these biases at scale.” 

The infamous Amazon resume screening tool scandal confirmed this when the tool penalized resumes containing the word “women” and prioritized male applicants for tech roles. In this case, even if the software wasn’t actively looking for gendered keywords, the AI tool’s biased training history led to “unconscious” systemic biases, much like we see in humans. 

C – Concealment problems 

Often called the “black box issue,” this problem is about how we really don’t understand the inner workings of AI resume scanners. In the Amazon example, recruiters could have assumed the tool’s selections were rooted in standard keyword logic, not bias, because they had no way of knowing why a candidate was accepted or rejected. 

This opacity can be problematic in two ways. 

Michael Weiss, partner at the Law Offices of Lerner & Weiss, warns us of the first: “When an AI makes hiring decisions, employers often can't explain the specific reasoning behind rejections, which is legally dangerous.”

Second, it undermines candidate trust – and understandably so. Pew Research Center found that 66% of Americans said they wouldn’t apply for jobs where AI was used in talent acquisition. The fallout is twofold: You miss out on top talent and damage your employer brand. 

Screening’s quiet evolution: AI resume scoring

There’s no denying that AI is invaluable in helping hiring teams process high volumes of applications and recruit at scale. But it’s now evident that traditional AI screening tools come with serious risks. 

That’s why disruptors in the recruitment tech space have now built a new kind of solution: AI resume scoring. This new system closes some of the gaping holes we discussed earlier, leading to faster, fairer AI resume screening.  

Hoegener defines AI resume scoring as “a way to rank resumes based on how closely they match what you’re looking for. You feed in the job requirements, and the system gives each resume a score depending on how well it fits those criteria.”

On the surface, this sounds similar to the traditional keyword model. But in practice, it works very differently. 

How AI resume scoring works

The best way to understand AI resume scoring is to see how it works step by step.

Step 1: Defining scoring criteria

First, recruiters add their job requirements – such as skills, knowledge areas, and experience levels – to the platform. The system uses this information to generate a set of suggested structured job criteria,  including technical skills, soft qualities, proficiency with tools, and more. 

These job criteria generate the standards for role-relevant skills that the tool will ultimately use to analyze and score the resume. So, it’s critical to get them right. That’s why some of the more advanced AI resume scoring tools allow you to create, edit, or delete job criteria yourself. 

For instance, if the AI suggests a criterion like “experience with design software,” you could edit this to something more precise, like “skilled in using industry-standard 3D animation tools to produce assets” if needed. This way, you’re in control of what the scoring rubric should look for.

Step 2: Uploading resumes 

After you’ve chosen your scoring criteria and your resume scoring assessment is live, candidates will be prompted to upload their resumes in standard formats like PDF or a Microsoft Word .docx file.

Before any analysis begins, the tool automatically removes personal identifiers – such as names, email addresses, photos, pronouns, and locations – to ensure the AI focuses entirely on skills and capabilities rather than irrelevant details that could bias analysis and decision-making. 

Step 3: Automatic scoring 

Once a resume is uploaded, the AI tool evaluates it against the job criteria you set in step 1. Different platforms vary in how they score. For instance, TestGorilla’s AI resume scoring assigns candidates a score from 0–5 for each of the five job criteria, with each criterion weighted equally by default.

It also provides a brief explanation of how it reached that score and lets you override scores if you need to account for additional context or new information.

In addition to the criteria-level scores, the tool also calculates: 

• A raw score, which represents the total alignment between the resume and the role. 

• A percentile score, which shows how candidates compare to others in the applicant pool. 

Pro tip: Test your setup before using it. You can upload a sample resume into the system, assess how it scores the candidate, and edit your criteria as needed before going live. 

Step 4: Decision-making 

After scores are finalized, you decide which resumes make the cut. You could, for instance, choose to shortlist just the top 10 resumes or review each resume score one by one, depending on the volume of applicants and your personal preference. 

Moreover, an AI resume scorer like TestGorilla’s is built into broader candidate assessment systems – with skills testing, interview tools, and more – so all your data is in one place. 

Head-to-head: Key differences between AI resume screening and AI resume scoring

Here’s how AI resume scoring differs from traditional AI resume screening tools:

Why AI resume scoring beats AI resume screening

Let’s take a look at how AI resume scoring bridges the gaps found in traditional screening models. 

Mitigates bias 

AI resume scorers automatically remove candidates’ names, pronouns, photos, and other identifiers from resumes before evaluating them. Research has shown that these blinding techniques are effective at minimizing bias. 

AI resume scoring tools also let recruiters test the system with a sample resume to ensure it’s working as intended before going live. This feature, combined with the regular fairness checks conducted on these tools, ensures the AI is audited for bias before it’s used to make decisions. TestGorilla’s science team, for instance, continually monitors its AI tool to ensure fair outcomes across demographic groups.  

Builds trust and transparency 

Unlike the opacity of the traditional “black boxes,” AI resume-scoring systems provide explanations for every score, showing exactly which parts of the resume align with each job criterion. This kind of transparency gives recruiters confidence that their hiring process is fair and rooted in evidence. 

Plus, it can put candidates at ease because instead of a mysterious rejection, you can provide them with tangible feedback about why they didn’t progress. 

Offers legal protection 

If a candidate challenges your hiring decision in court (for example, by claiming unfair rejection), you need a clear audit trail to defend yourself. AI resume scoring tools record the exact criteria used and how each resume was evaluated; this helps you demonstrate that decisions were based on objective, job-related factors rather than arbitrary or biased ones. 

Keeps humans in the loop 

AI resume scoring isn’t entirely hands-off. It gives recruiters control at every stage of the screening process, from setting job criteria to overriding scores when needed. “Human oversight is EVERYTHING. You just can’t trust AI to understand context,” Hoegener tells TestGorilla – and we agree. 

Hickey helpfully captures why you can’t go all-in on AI: “Take, for example, the case of a candidate who took two years off to care for a sick relative. AI sees an employment gap; a human sees compassion and responsibility – leadership traits, not red flags.” 

A human touch is critical because humans can put candidates’ experiences into context, spot when AI-generated resumes fabricate skills (for instance, by noting misalignment), and identify the softer qualities that make a candidate stand out. 

Helps with accurate and holistic screening decisions 

Let’s face it, the resume itself is a flawed tool because it’s not verifiable. As Hoegener rightly says, “The problem is, just because a resume scores high doesn’t mean the person is actually qualified.” And, considering 70% of candidates admit to lying on their resumes, it makes it even scarier. 

This is where AI resume scoring tools built into a broader assessment platform shine. 

With TestGorilla, for example, candidates are first put through talent assessments to test their hard and soft skills, cognitive abilities, personality traits, and values. Then, they upload their resumes for scoring. That means you factor in their real-life test scores plus resume scores when making decisions. 

In fact, research for our 2025 State of Skills-Based Hiring report revealed that 98% of employers combine resume screening with skills tests. And those who used skills tests first were happier with the quality of their hires: 96% vs. 87% for those who screened resumes first. 

AI resume scoring is restoring our confidence in responsible screening

AI resume scoring tools are changing how we use AI at the screening stage. With job-specific scoring criteria, fully explainable scoring systems, regular bias checks, and humans firmly in charge, these new tools blend technology with judgment in the best possible way. This brings fairness, clarity, and control back into a process that had slipped out of our hands. 

Check out TestGorilla’s AI resume scoring tool and book a free live demo to see it in action.   

Contributors

• Friddy Hoegener, SCOPE Recruiting, Co-Founder

• Jim Hickey, Perpetual Talent Solutions, President and Managing Partner

• Michael Weiss, Law Offices of Lerner & Weiss, Partner

• Pankaj Khurana, Rocket, VP Technology & Consulting