Why “uses AI daily” is the wrong bar

Most organizations have defined AI fluency at the tool level, i.e., “X candidate knows that tools exist and for what purpose”. That's the root of the problem, because a tool-level definition can't cross functional lines. If your definition of fluency is “knows ChatGPT,” it falls apart the moment you try to apply it to a customer service rep and a performance marketer in the same breath. 

Here's how scattered the bar really is. When we asked hiring managers where they set the minimum threshold for AI fluency, the answers didn't cluster around a standard. They split four ways:

Four different bars, all wearing the same “AI-fluent” label. When the term means different things to different organizations, the rubric can't travel, the benchmark can't hold, and the hire that looks right on paper looks wrong on the job.

The fix isn't a sharper buzzword but a definition built on behaviors you can observe, not relationships with AI you can only take on faith. Industry terms we’ve heard recently like “AI-ready” tells you a candidate is willing. “AI-friendly” tells you they're open. Neither tells you whether they can deliver. You can't score “willingness” against a consistent measurement.

What AI fluency looks like, function by function

The behaviors that signal fluency are constant across roles. Judgment over outputs, ethical awareness, and the ability to communicate AI use to colleagues who need to understand or audit it stay the same across functions. What changes is the tool, the risk profile, and the vocabulary.

So the right way to define fluency for a non-technical role is to start from the behavior and translate it into that function's daily work. Here's that translation for five roles where AI adoption is already high and assessment is still mostly guesswork.

Notice the pattern. The weak-signal column is always a tool plus a task. The fluency column is always a judgment: knowing when to automate, when to augment, and when to step back from AI entirely.

The five behaviors underneath every role

Those role-specific translations all trace back to the same five dimensions. The following is our own internal framework that we use every day at TestGorilla and was built by our Talent and Assessment Science team. It’s grounded in IO psychology research and validated against job performance data, specifically so it generates observable signals across technical and non-technical roles alike.

The five pillars are:

1. Applied AI use and workflows: this is what people most commonly think of when thinking of “AI use”. It’s selecting the right tool, breaking work into AI-suitable parts, executing real work end to end, and explaining the trade-offs. Guards against the candidate who describes AI beautifully but can't operate it under real conditions.

2. Learning and digital agility: keeping up as the tools and models churn. The skill that stays valuable when this year's tool is next year's footnote.

3. Systems thinking and problem solving: reasoning about downstream impact, spotting risk, and designing verification into the workflow before something breaks two steps later.

4. Responsible and ethical AI use: recognizing bias, privacy, and governance constraints. In our data, this ranked as the most frequently cited number-one skill hiring managers look for, ahead of even applied tool use.

5. Human-AI collaboration and communication: documenting how AI was used so colleagues can review, learn from, and build on it. The difference between fluency that scales across a team and fluency that walks out the door when one person leaves.

There's no universal ranking. No one pillar matters more than the other, and that's deliberate. It all depends on your particular business, morals, and goals. This framework is meant as a diagnostic tool, not a fixed hierarchy. 

The right weighting depends on your industry, your organizational stage, and the specific role. A regulated healthcare or legal team will weigh ethical use heavily. An early-stage startup will lean toward agility and experimentation. The point of the framework isn't to force one weighting. It's to make sure no critical dimension gets quietly skipped, which is exactly what happens when assessment collapses into tool familiarity and interview confidence.

The beauty of the five pillars is that when it comes to making a hiring decision, they evolve the questions you ask from something like “do they use ChatGPT?” into three much more practical ones:

1. Can they deliver with AI?

2. Can they be trusted with AI? 

3. Can they help others reach their level?

If the answers are “Yes” they can be considered to be AI-fluent. 

How to assess AI fluency

Here's the uncomfortable part. A standard interview is built to observe communication: what a candidate says, how they frame it, and how confidently they hold a position. It is not built to observe execution. And for AI fluency, the gap between those two things is where almost all bad hires originate.

Here are four ways to make your hiring process better at assessing AI fluency: 

Demonstration over self-report. Candidates know what you want to hear, and under a competitive job market, they'll report whatever gets them through the screen. That's not dishonesty, it's rational behavior in response to a badly designed evaluation. The fix isn't to ask sharper questions about their self-reported usage. It's to stop treating self-reports as evidence and require a demonstration of using AI instead.

Scenario over recall. The highest-leverage change most teams can make is to require candidates to do something with AI before the decision is made. It doesn't need to be elaborate. A scenario prompt built around a core work task for the role, scored against a rubric that rates execution, judgment, and verification, will surface more signal than a full interview panel built on conversation.

Confidence vs competence. The most dangerous proxy in AI hiring is fluency of speech. A candidate who discusses AI concepts smoothly will score well on most current rubrics, because most current rubrics are built around conversational signals. Confidence without specificity isn't a signal; it's noise, and your process has to be designed to tell the difference.

Adaptation over preparation. Static tasks produce static signals. The most predictive assessments change the conditions mid-task. Remove a tool, add a compliance constraint, or shift a requirement partway through, and watch whether the candidate reframes or stalls. As our Director of Talent and Assessment Science, Romina da Costa, puts it: “the psychometric signal isn't the output the candidate produced. It's the metacognitive process: how they got there, what they chose to verify, and how they adapt when the technology won't sit still.”

The questions that surface genuine capability

You don't need to rebuild your hiring process to apply all of this. You can start by changing one question. Tool-name questions are the most common AI hiring questions and the least predictive, because they measure exposure, not capability. 

Swap them for questions that demand specifics no one can fake without having done the work.

The logic, borrowed from performance-based hiring, is to go narrow and deep rather than broad and shallow. Have you used AI to redesign a workflow? What changed? What broke? What did you verify? What would you do differently? Those questions can't be answered convincingly by someone who hasn't actually done the work.

Score it consistently with one simple lens

A question is only as good as the rubric behind it. Without a shared scoring standard, you're back to manager discretion, which 19% of organizations still rely on with no anchor at all. The five pillars give you scoreable dimensions; a three-point scale keeps them usable.

Score each candidate on execution (pillar 1), adaptability (pillar 2), systems thinking (pillar 3), judgment (pillar 4), and communication (pillar 5). Decide the weighting before anyone enters the process, based on the role. For a compliance-heavy finance or legal hire, judgment might be non-negotiable while strategic optimization stays developmental. For a growth marketer, you might flip that. The framework holds either way.

Where to start

The shift from guesswork to evidence doesn't require a full overhaul. Pick one open role and run a structured, scenario-based AI fluency assessment alongside your existing process. Track the outcomes. Compare the hire quality. The organizations that have done this don't go back.

For non-technical roles specifically, this is where proprietary test coverage earns its place. TestGorilla's AI fluency assessments are built directly on the five pillars and move past multiple-choice theory into scenario-based tasks and behavioral evaluations that generate consistent, auditable signals across marketing, HR, finance, legal, and operations, not just engineering. Every score is explainable rather than a black box, so you get a breakdown you can actually defend.

The candidates who can deliver with AI, be trusted with it, and bring their teams up with them are out there. The only real question is whether your hiring process is built to find them, or just to be impressed by the ones who talk about it well.

Ready to test for it, not just talk about it? Build your first AI fluency assessment for a non-technical role and see what your candidates can actually do.