Why 78% of candidates choose AI job interviews (and what it means for hiring)

The results are in: AI job interviews lead to improved hiring outcomes

First, let’s discuss research conditions and outcomes.

In the study at the Booth School, experiment leads allocated participants into one of three treatment conditions:

1. Randomized AI interview group, where participants were interviewed by an AI voice agent. 

2. Randomized human interview group, where participants were interviewed by human recruiters. 

3.  Choice group, where participants could choose how they wanted to be interviewed – by an AI or a human. 

In all three conditions, human recruiters assessed interviews and made final hiring decisions based on candidates’ interview performance and scores on standardized tests. 

Fast facts: What recruiters predicted 

• 61% of recruiters expected AI interviews to be lower quality than human interviews. 

• 36% of recruiters expected AI interviews to lead to lower offer rates, 49% said they would be equal, and 15% expected job offer rates to be higher for AI interviews. 

• 48% of recruiters expected retention rates to be lower for AI interviews, 39% said they would be equal, and only 13% expected higher retention rates for AI interviews. 

Fast facts: What the numbers say

• 78% of candidates in the “choice group” chose the AI voice agent over a human interviewer. 

• Job offer rates were 12% higher for AI interviews (9.73%) compared to human interviews (8.70%). 

• Start rates (percent of the total pool who started jobs) were 18% higher for AI interviews (6.62%) compared to human interviews (5.63%).   

• 30-day retention rates were 17% higher for AI interviews (5.42%) compared to human interviews (4.62%). 

• Among candidates who accepted offers, start rates and 30-day retention rates were each 6% higher for the AI interview group. 

The takeaway 

In short, the data told a very different story from recruiters’ initial predictions. AI interviews led to more job offers, but importantly, higher start and retention rates, indicating better quality matches. 

The question is, why? 

Diving deeper: AI improved the hiring process itself 

Here are the main reasons AI interviews performed better than human interviews:

AI actually conducted higher-quality interviews

For example, more AI interviews were considered more “comprehensive” (42% versus 39% of human interviews) and had a better structure, with a clear beginning, middle, and end. 

They also covered more key hiring-relevant topics on average (6.78 out of a maximum of 14 versus 5.53 for human-led interviews). 

Candidates’ language was more impressive in AI-led interviews  

On the language side, AI interviews scored higher for factors that typically predict successful hiring outcomes, such as more conversational exchanges, better candidate vocabulary, and fewer filler words.

Recruiters rated AI interviews higher

Recruiters assigned significantly higher interview scores to applicants interviewed by the AI voice agent (average score of 2.01) compared to those interviewed by humans (average score of 1.90). 

Recruiters also showed a more positive sentiment towards these interviews during their qualitative assessments. Sixty-nine percent of recruiters had a higher chance of making an offer when the interview was AI-led. 

AI interviews made hiring cheaper and more efficient 

Interestingly, the study found that using AI cut interview costs by almost half ($1.30 per AI interview versus $2.48 per human-led interview). For high-volume hiring, that’s a meaningful saving. 

That said, the overall time-to-hire was higher for AI interviews: a median of 22 days versus 19 days for human-led interviews. However, this was attributed to the recruiters taking longer to review AI interviews and make final hiring decisions. 

My opinion? This isn’t necessarily a bad thing because recruiters are still spending their own time on more meaningful and strategic tasks, such as evaluating interviews and selections. 

In fact, the study found that using AI significantly reduced the time spent on manual tasks, such as scheduling interviews. It took a median of 0.32 days from application to interview with AI, versus 0.51 days with humans. 

Wynter Johnson, Founder and CEO of Caily, agrees that using AI is an excellent way to save time and energy when dealing with high application volumes. She tells me, “AI-led interviews are a great screening tool to help cut through [...] applications. It’s an arms race between candidates and companies these days, and AI interviews can give companies an edge.”

But why do candidates prefer AI interviews? Peeling back the layers

The study also revealed a strange paradox. After the interviews, when candidates were asked, “How natural did the interaction with the recruiter feel?” those interviewed by the AI voice agent rated the interaction as significantly less natural than those who spoke to human recruiters. 

But this perceived lack of naturalness didn’t translate into a poorer candidate experience. (Applicants rated variables like perceived stress, comfort, follow-up fluency, and feedback quality similarly across both treatment conditions.)

Additionally, the Net Promoter Score (NPS), which measures the likelihood of candidates recommending the company to others based on their experiences, was “almost identical across treatments.”

If anything, the candidates favored AI interviews, despite rating the interaction as less natural: 71% of applicants provided positive feedback about the interviews in the AI-led group, compared to 52% in the human-led group. 

So, why did candidates not only choose AI interviews but also slightly prefer them? 

Positive feelings about AI in general

A survey conducted as part of the study revealed that participants generally had positive attitudes toward AI. For example, 47% of them believed AI would have a positive personal impact on them in the workplace. This attitude also predicted their choice, as 77% of respondents who were generally optimistic about AI chose the AI voice agent interview. 

This suggests that most candidates don’t necessarily view AI as a threat and are comfortable with its use in hiring. Compare this to just two years ago, when research by Pew showed that 66% of individuals wouldn’t even apply to companies using AI in hiring. 

Preferences for speed, convenience, and consistency

Practicality and speed were two significant draws for candidates who chose to be interviewed by AI. 

Qualitative discussions revealed that candidates in the AI-led interview group felt they could schedule their interviews at a time that suited them, including immediately after receiving the invitation. Meanwhile, the other group, where human recruiters’ availability issues could delay the process, felt they couldn’t. 

Furthermore, candidates interviewed by AI also took comfort in a consistent and predictable interview format, which isn’t always the case in human-led interviews that can go off on a tangent. 

My conversation with Emma Joy Iezzoni, Consulting Executive at EJ Intelligence and Adjunct Professor at Villanova School of Business, adds further depth to this. Iezzoni points out that generational factors could be driving some of these preferences:

“It’s partly generational: This is the same group that would do almost anything to avoid a phone call. For many candidates, self-service through AI feels less intimidating, more efficient, and more transparent than navigating the social dynamics of a live recruiter.”

Lower social pressure 

Johnson also points out something interesting: AI interviews tend to cause less social pressure and angst for candidates than human-led ones. “You won’t have people reacting to your body language, your pauses, your outfit, or your appearance,” he says.

I found this same sentiment echoed through candidates’ post-interview surveys in the Booth School study, where 45% of candidates in the AI group used words like “comfortable” and “positive” when describing their interview experience, compared to only 19% of the human-led interview group. 

The study’s linguistic analysis supported this. There were fewer instances of filler words and backchannel cues, such as “uh-huh” or “yeah," in AI-led interviews. This suggests that candidates were more confident and focused, with less of the subtle pressure that comes with reading another person’s reactions. 

Perceived fairness and reduced bias

Unconscious bias is well-documented as one of the most significant issues in interviews. However, the field experiment’s post-interview survey revealed that significantly fewer participants interviewed by AI voice agents reported experiencing gender bias (3.30%) compared to those interviewed by humans (5.98%) – a meaningful difference in how candidates perceive fairness across treatment groups. 

Of course, humans can be swayed by other interviewer biases, such as in-group biases, where they favor candidates who remind them of themselves (for instance, those who support the same sports team). 

They can also be influenced by the halo effect, where a single credential colors an interviewer’s entire perception of an applicant – for instance, perceiving an Ivy-League candidate’s responses as better than they truly are. 

While the study doesn’t inspect for these biases, Iezzoni puts it all into perspective, telling me, “AI can help shift the focus from resumes to real responses. As David Brooks has argued, fancy schools don’t always equal professional success.”

When candidates believe that AI can strip away some of these unconscious biases, they’re more likely to view AI interviews as a fairer playing field. 

Perceived lower barriers to success

Lastly, the study uncovered a subtle but more complex reason behind the AI interview’s popularity. Applicants who chose the AI voice agent scored lower on the language test (3.14 vs. 3.37) and the analytical test (47.54 vs. 49.77) compared to those who picked a human interviewer. 

These findings suggest that candidates who were objectively less skilled might have viewed the AI interview as a less stringent barrier or something they could better “game” compared to human interviewers. 

Still, AI interviewing has flaws

AI isn’t flawless. That doesn't mean you shouldn’t use it, especially when the data is strongly in favor of it. But you must be aware of its downsides. 

I spoke to several industry experts and reviewed other pieces of research to explain why entrusting your entire interview process to AI is a step too far. 

AI isn’t immune to biases and errors

Even in the Booth School study, roughly 7% of interviews failed due to AI system breakdowns. 

While technical difficulties are easier to fix, the real risk with AI is that it’s not 100% free from biases. It can perpetuate human biases and land you in some serious legal trouble. Michael Weiss, a lawyer with more than 40 years of experience representing employers, shares a firsthand example of this:

“I recently defended a client where their AI interview platform consistently screened out candidates over 45, even though age wasn't an explicit factor. The algorithm learned from historical hiring data that favored younger employees, creating a textbook disparate impact case.”

Weiss doesn’t advise against using AI in interviewing. Rather, he advocates for humans to be in control of and able to explain final hiring decisions, thereby avoiding a “legally dangerous” position. 

In the Booth School Study, for instance, recruiters not only used but also placed greater weight on standardized tests when making hiring decisions for candidates interviewed by AI. According to Wahal, “having this additional data point helps to significantly reduce the risk of making biased or incorrect hiring choices they can't explain.”

Risk of some candidates gaming the AI system 

Participants in the Booth School study who had poorer test scores tended to gravitate more toward AI, perhaps perceiving it as a more accessible option. But this brings an even bigger concern to the surface – one that Iezzoni articulates well.

“The biggest risk is an ‘AI loop’ – candidates using AI to craft answers while AI is screening them,” she tells TestGorilla. “Research (see Ethan Mollick’s work) shows this breaks down quickly without a human in the loop. Recruiters need to be thoughtful about where machine judgment ends and human judgment begins.”

It’s important to acknowledge that most candidates don’t cheat, but among those who do, AI is the go-to tool. Our research found that while only 17% of candidates cheated during the hiring process, 70% of them used AI to do so.)

That’s why it’s crucial to use tools like skills assessments alongside AI interviews to ensure a fair, objective assessment. Recruiters must also thoroughly audit the tools they use, maintain oversight in reviewing AI interviews, and closely monitor for any signs of cheating. 

Lucy Dewhurst, our very own Content and Communications lead, has an anecdote about why it’s not as easy as it seems: “We found a recent candidate was reading aloud real-time answers from an AI bot as I asked the questions – which was a new one and hard to spot unless we really listened.”

Some AI tools fail at assessing soft skills

Based on the World Economic Forum’s latest Future of Jobs report, soft skills like creativity, resilience, and curiosity are among the most highly sought-after capabilities for the next five years – across all industries and job roles.

Speaking to TestGorilla, Dr. Matthew Casavant, a Physician and Business Owner at South Lake OB/GYN & Advanced Surgery, emphasizes the importance of these human skills: “The employees who excel aren’t necessarily the ones with perfect interview answers – they’re the ones who naturally focus on patient comfort during those vulnerable moments.”

Unfortunately, soft skills and traits are easily misjudged by poor-quality AI interview tools.

Research supports this view. MIT Technology Review tested one AI platform and found that, in addition to scoring a German speaker highly on English competency, the platform also scored candidates’ personalities based not on what they said, but on their vocal intonation. 

“We really can’t use intonation as data for hiring,” Fred Oswald, a psychology professor at Rice University, explains to MIT Technology Review. 

“That just doesn’t seem fair or reliable or valid,” Oswald adds, again pointing to the importance of having the right checks and balances in place and pairing AI interviews with more accurate tools like skills assessments. 

Candidates still want (and need) a human touch 

While we’ve seen that candidates tend to prefer AI interviews, 5% of the Booth study’s participants outright refused to engage with an AI voice agent, showing extreme aversion. 

Although this is a small percentage, previous research has shown that a larger proportion – 7 in 10 Americans – oppose the use of AI in choosing who gets a job. 

This aligns with everything the experts have said so far, as well as our view here at TestGorilla, that the final hiring decision should always be made by humans. 

Wahal also notes that, beyond all this, candidates still need human interaction to learn about the company and its leaders, as well as to ask questions:

“If AI runs the entire interview process, when do candidates get the chance to understand the people, culture, and values behind the job offer? In my opinion, AI job interviewing stands on solid ground, but more as a replacement for screening calls and first-round interviews, not advanced or final-stage interviews.”

Getting ahead of the curve: Safe ways to test AI interviews out this quarter

We’ve unpacked a lot here. On the one hand, we’re seeing a very strong use case for AI interviews. On the other, experts are recommending caution with over-reliance on AI.

So, what can you do with all this information? If you ask me, the smartest employers are already testing the waters themselves. 

Here’s how you can pilot AI interviewing in your company in a safe and ethical way:

• Use AI interview tools only for early-stage screening, not high-stakes mid-stage or final interviews.

• Pair AI interviews with skills assessments to get a broader set of data points that reduce bias, prevent cheating, and assess soft skills. Bogdan Radičević, Measurement Scientist at TestGorilla, says, “AI interviews are not a silver bullet; they are one powerful tool in a recruiter's arsenal. Combining them with other sources of information (e.g., skills, cognitive, [and] personality tests) can provide a truly holistic picture of a candidate.”

• Pick software like TestGorilla’s AI interviewing tool. It enables you to conduct in-depth screening interviews with structured prompts, bias testing, and transparent scoring, while reviewing and selecting candidates remains in your hands. Plus, you can access 350+ talent assessments for a complete evaluation on the same platform. 

• Track outcomes like offer rates, retention, time-to-hire, and cost-to-hire to monitor how AI interviewing is impacting your recruitment process. 

• Be upfront with candidates about how you’re using AI to reduce AI anxiety and aversion. Sanja Sreckovic, Head of Recruitment at DesignRush, advises that “candidates must understand the ways AI is used, and there needs to be a human component to the process.”

Caveats and context: Limitations of the study

The study conducted by the Booth School of Business used a sample of approximately 70,000 applicants for call-center customer service roles in the Philippines. Hiring outcomes were measured at the 30-day point only, indicating that the results could have been short-term. 

The researchers also documented negative sorting, as candidates with lower test scores were more likely to pick the AI condition. 

Future research should track longer-term performance outcomes and expand to other industries and geographies to test the transferability of these results.

Contributors

• Bogdan Radičević, TestGorilla, Measurement Scientist

• Emma Joy Iezzoni, EJ Intelligence, Consulting Executive

• Lucy Dewhurst, TestGorilla, Content and Communications Director

• Dr. Matthew Casavant, South Lake OB/GYN & Advanced Surgery, Physician & Owner

• Michael Weiss, Law Offices of Lerner & Weiss, Partner

• Sanja Sreckovic, DesignRush, Head of Recruitment

• Wynter Johnson, Caily, Founder and CEO

• Yashna Wahal, HR and hiring expert