How to hire an AI engineer

What does an AI engineer actually do?

They put models into production and keep them working. That sounds obvious until you try to write the requisition, at which point three quite different jobs turn out to be wearing the same title.

Most AI engineering roles take one of these shapes. Hiring well starts with knowing which one fixes the problem you actually have:

A useful rule circulating among people who hire for these roles: hire for the failure mode you already have. If your prototypes never survive real traffic, you need infrastructure, not a researcher. If the model performs well and nobody uses the feature, you need product judgment, not a better model.

Founders often ask for one person who does all three. That person doesn’t exist, costs a fortune, and usually does not want the job you’re hiring for. It’s better to define the primary responsibility and name the gaps you will tolerate.

What's the difference between an AI engineer and an ML engineer?

A machine learning engineer trains, evaluates, and deploys models. An AI engineer builds systems on top of models someone else trained. The ML engineer's core skill is the model. The AI engineer's core skill is everything around it.

The distinction matters commercially. Most companies adopting AI in 2026 do not need anyone to train a model. They need someone who can take a capable general model and build something dependable with it: retrieval that returns the right document, prompts that hold up under strange inputs, fallbacks for when the model fails, and a way to know whether the whole thing is getting better or worse.

If you write an ML engineer requisition when you need an AI engineer, you will interview people who can discuss loss functions and cannot tell you why your feature times out. The reverse mistake is rarer but just as expensive.

Why is it so hard to hire AI engineers?

The market gives one answer: scarcity. Small pool, enormous offers from large labs, candidates who will not answer generic outreach. All of that is real, and none of it is the reason most AI engineering hires go wrong.

The harder problem is that you cannot tell who is good, and there isn’t much history for these types of roles to look at. Meanwhile, signals that used to work, like the resume and interview, have quietly been broken. 

A resume listing Python, PyTorch, and LangChain tells you a candidate knows the buzzwords. A GitHub repo with a working retrieval demo tells you slightly more than it used to, which is to say not much: a competent generalist with an AI assistant can produce that in a weekend. The artifacts got cheaper to make faster than our evaluation methods got better.

Our own research suggests how widely this is felt. In our recent State of Hiring for AI Fluency report, in which we surveyed around 2,000 senior hiring leaders in the US and UK, 53% said they now prefer a candidate with high AI fluency over one with deep domain expertise. Yet 59% of those same organizations have still made a bad AI hire. 

Compensation makes the stakes worse rather than solving anything. Senior AI engineering salaries are high, agency placement fees commonly run a fifth to a quarter of first-year cash compensation, and an eighteen-month tenure is a normal outcome. A mis-hire in this role is one of the more expensive mistakes available to a mid-sized company.

What skills should you look for in an AI engineer?

Not a tool list. The tool list is the part that has stopped working, and screening a resume for framework names is close to noise now. 

We’ve determined five capabilities that are worth evaluating directly, and all five can be observed so that you can come away from a candidate interview feeling confident. 

1. Production judgment: the demo-to-production gap

The distance between something that works in a notebook and something that survives real traffic, real cost, and real failure. A candidate with production judgment talks unprompted about what happens when the model returns nonsense, when the API is down, when a user pastes in something enormous, and when the bill arrives. A candidate without it demos the happy path and stops.

2. Eval literacy

Can they design a test that tells them their own system is getting worse? This is the closest thing to a hard differentiator in AI engineering right now, and it is the skill most hiring processes never check. Anyone can tell that a system works today. Knowing whether last week's prompt change quietly degraded twelve percent of answers requires deliberately building the means to find out. Ask what they measured, how, and what the measurement caught.

3. Build, buy, or fine-tune

Model and tool selection is a judgment call with real money attached. Fine-tuning when a better prompt would do wastes months. Prompting when the task genuinely needs a trained model produces something that never quite works. Strong candidates reason out loud about the tradeoff and can name a time they chose the boring option.

4. Cost and latency awareness

An AI feature that costs more per call than the revenue it generates is a failed feature, however clever. So is one that takes eleven seconds to respond. Engineers who have run something in production carry these numbers around in their heads. Engineers who have only built prototypes have usually never looked.

5. Responsible deployment

Guardrails, data handling, and knowing which decisions a model should not make on its own. Where does user data go when it passes through a third-party model? What happens when the system is confidently wrong in front of a customer? This is not a compliance checkbox. It is the difference between an incident you designed for and one that finds you.

Where do you find AI engineers?

The strong ones are employed, selective, and buried in generic outreach. Volume recruiting works badly here. The goal is fewer, better conversations, which means going where the work is visible rather than where the applicants are.

• Open-source contributions to AI tooling. Anyone contributing to an inference library or an eval framework is demonstrating exactly the capability you want to buy.

• Model repositories and eval leaderboards, where published work carries a method you can inspect.

• Technical writing. An engineer who has explained a hard tradeoff in public has shown you their reasoning for free.

• Referrals from engineers you already trust, which remain the highest-yield channel in this market and the one most teams underuse.

Whatever the channel, the point is the same: look for evidence of decisions, not evidence of exposure.

How do you assess an AI engineer's skills?

Watch them work on something that resembles the job. That principle is not a preference; it is one of the better-established findings in personnel selection: work sample tests and structured interviews consistently outpredict resumes and years of experience, while years of experience lands near the bottom.

For an AI engineering role, a defensible process usually has three parts.

Start with a practical work sample that mirrors your actual failure mode. Give them a system that is subtly broken, or a feature with an obvious happy path and three unhandled edge cases, and watch what they notice first. What a candidate investigates before writing code tells you more than the code does.

Then use structured skills tests to check the underlying technical foundation, so you are not inferring everything from a single exercise. Pair them rather than relying on one, since no single measure should carry a hiring decision. Relevant options include the Artificial Intelligence test and the Machine Learning test, combined with language-specific coding tests and a cognitive ability measure for the reasoning this role demands under ambiguity.

Finish with a structured interview that pressure-tests the judgment behind the scores, using the questions below. If you would rather run that stage consistently across a panel, our AI Engineer interview covers model design and deployment, validation, optimization, and technical communication. Keep the total ask proportionate either way. A 30 to 60 minute assessment plus interviews is reasonable. A multi-day take-home will lose you the candidates you most wanted.

It’s worth noting how rare any of this still is. Our aforementioned report found that only 26% of organizations currently require candidates to demonstrate independent AI use and verify the results during hiring. 

Most companies are still taking AI capability on trust, at exactly the moment trust stopped being informative.

What should you ask an AI engineer in an interview?

Five questions that surface judgment rather than vocabulary. For each one, the useful part is not the answer's polish but whether it contains a specific decision the candidate actually made.

"Tell me about an AI feature you shipped that broke in production. What broke, and what did you change?"

Strong answers are specific and slightly unflattering: a real failure, a diagnosis, a fix, and something they now do differently. Weak answers describe a project that went well, or blame the model. Engineers who have run things in production always have this story.

"How would you know if a change you made quietly made the system worse?"

This is eval literacy in one question. You want to hear about a measurement they built or would build, and what it would catch. Vague answers about testing carefully mean they have never had to find out.

"Walk me through a time you chose not to use the more sophisticated approach."

You are listening for a reasoned tradeoff: prompting instead of fine-tuning, a smaller model, an off-the-shelf tool instead of building. Candidates who cannot name one have usually optimized for interesting rather than shipped.

"What does this feature cost to run, and how do you know?"

Ask it about something they built. Strong candidates know their per-call costs and latency numbers, or know exactly where to look. Blank responses tell you they have built prototypes, not products.

"Where in this system would you refuse to let the model decide on its own?"

A good answer names a specific decision and a specific consequence, then describes the guardrail. Candidates who say the model handles everything, or who escalate everything, are both telling you their judgment is uncalibrated.

Hire for what they can show you

AI engineering hiring has an evaluation problem dressed up as a scarcity problem. The pool is genuinely tight, but the reason good candidates slip past and weak ones get through is that the old proof stopped working. A tool list, a resume, and a working demo are all now cheap to assemble.

What has not gotten cheaper is judgment: knowing what breaks, what it costs, and how you would find out. Screen for that and the tight market matters less than you think.

Build an AI engineer assessment from 400+ scientifically validated tests, or see how TestGorilla measures AI fluency across technical and non-technical roles. Try it free.

Frequently asked questions

What is the difference between an AI engineer and an ML engineer?

A machine learning engineer trains, evaluates, and deploys models. An AI engineer builds systems on top of models that already exist, handling retrieval, orchestration, prompting, guardrails, and failure handling. Most companies adopting AI need the second role, not the first, because they are building on general-purpose models rather than training their own.

What skills does an AI engineer need?

Production judgment and eval literacy first: the ability to build something that survives real traffic, and the ability to measure whether it is getting worse. Those sit on top of a technical foundation in Python, model APIs, retrieval, and deployment. The specific frameworks matter far less than they used to, because tools change quickly and naming them proves little.

How do you test an AI engineer's skills before hiring?

Use a work sample that mirrors your real failure mode, paired with structured skills tests and a structured interview. A portfolio or demo alone no longer discriminates, because AI-assisted coding makes a working prototype cheap to produce. What you need to observe is how a candidate reasons about failure, cost, and measurement.

Do you need an AI engineer or a software engineer who understands AI?

Often the latter, and it is worth asking honestly before you open the requisition. A strong backend engineer who has shipped against model APIs can handle retrieval, orchestration, guardrails, and evaluation for most business use cases. That pool is far larger and less expensive than the research-adjacent one, and for many companies it is the better hire.

How much does it cost to hire an AI engineer?

Senior AI engineering compensation runs high, and agency placement fees commonly fall between a fifth and a quarter of first-year cash compensation. Treat published salary figures cautiously, since they move quickly and vary sharply by market and role shape. The larger cost is usually the mis-hire rather than the salary.