BlogAI adoption

Why healthcare AI pilots stall, and what to check first

Tyler Kent · November 16, 2026 · 6-minute read

Immature AI tools topped the list when 43 large US health systems were asked what was holding back their AI projects. In that survey, run in fall 2024 among nonprofit health systems and published in the Journal of the American Medical Informatics Association (JAMIA) in May 2025, 77% named immature AI tools as one of their two biggest barriers to developing or deploying AI, ahead of financial concerns (47%) and regulatory or compliance uncertainty (40%).

77%

of 43 US nonprofit health systems named immature AI tools one of their two biggest barriers to AI, fall 2024

In a fall 2024 survey of 43 US nonprofit health systems, published in JAMIA in May 2025, 77% named immature AI tools as the biggest or second-biggest barrier to developing or deploying AI.

Source: Poon and others, Adoption of artificial intelligence in healthcare: survey of health system priorities, successes, and challenges, Journal of the American Medical Informatics Association.

KentAppliedkentapplied.com/facts/health-systems-immature-ai-tools-top-barrier-2024

Research across all industries names other causes. Gartner predicted in July 2024 that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, and it named four causes: poor data quality, inadequate risk controls, escalating costs and unclear business value. Most of these causes can be checked before a pilot is funded, and the six checks near the end of this piece show how.

How many organizations get an AI tool into production?

Few, by the count in MIT NANDA's 2025 report on AI in business, which sets a high bar for production. Of the organizations in its research, 60% evaluated custom or vendor-sold generative AI tools, 20% reached a pilot and 5% reached production. The report covers healthcare among other industries and rests on interviews with 52 organizations, a survey of 153 senior leaders and a review of more than 300 public AI projects.

It counted a tool as in production once users or executives described a marked and lasting effect on productivity or profit. The report presents its results as preliminary findings and calls these figures directionally accurate, because they come from interviews rather than from company records, and it adds that success definitions may differ across organizations.

Fig. 1

Only 5% of organizations put a custom or vendor-sold generative AI tool into production

Share of organizations in MIT NANDA's 2025 research that evaluated, piloted or put into production a custom or vendor-sold generative AI tool

Show the numbers as a table
Only 5% of organizations put a custom or vendor-sold generative AI tool into production
CategoryValue
Evaluated a tool60%
Reached a pilot20%
Reached production5%

How far have health systems taken AI?

Few have taken it across the whole organization. The consulting firm Deloitte surveyed 180 top executives at large health systems in the United States and five other countries in August and September 2025, for its 2026 Global Health Care Outlook. About 30% of the health systems reported running generative AI at scale in select areas of the organization, and 2% reported deploying AI across the entire organization.

A year earlier, the JAMIA survey found a similar gap in the six uses of AI in its clinical documentation and chart review categories. Ambient notes, AI that turns the conversation between a clinician and a patient into a draft note, was furthest along: 6 of the 43 health systems (14%) had deployed it fully and 20 (47%) had deployed it in limited areas. The authors describe ambient notes as rapidly advancing, and every health system that answered had at least begun developing or piloting it, which no other use in the survey matched.

The other five uses were further behind. One health system had fully deployed AI for abstracting data for clinical registries, none had fully deployed any of the other four, and 57% to 74% of the health systems that answered were still developing or piloting each of the five. Other kinds of AI were further along: 48% of the health systems had fully deployed AI for early detection of sepsis, and 40% had fully deployed AI for imaging and radiology.

Fig. 2

Ambient notes had spread, but full deployment of AI for notes and charts was rare

Stage reached by US nonprofit health systems for the six AI uses in the survey's clinical documentation and chart review categories, share of the 41 to 43 that answered for each use, in percent, fall 2024. The rest reported no activity.

Show the numbers as a table
Ambient notes had spread, but full deployment of AI for notes and charts was rare
CategoryDeployed fullyDeploying in limited areasDeveloping or piloting
Ambient notes14%47%40%
Abstracting data for registries2%10%57%
Inpatient chart summaries0%7%71%
Clinic chart summaries0%7%74%
Ambient nursing0%2%67%
Drafting care plan notes0%2%68%

Why do AI pilots stall?

The sources point to the tools themselves and to how organizations prepare for them, but they weigh the causes differently. The health systems in the JAMIA survey put immature tools first, and the authors write that "fit-for-purpose and practical evaluations are necessary before an AI tool is deployed, during its implementation, and in an ongoing way," so that a tool's benefits keep justifying its costs and risks.

Gartner's four causes are as much about money and management as about technology. Its analyst Rita Sallam said that with generative AI, "costs aren't as predictable as other technologies." Deloitte writes that health systems are under increasing pressure to show a clear return on their AI. In its survey, 51% of respondents said they had not measured the returns or that it was too soon to see results, while 31% reported "moderate" financial returns and 3% "significant" ones.

MIT's report puts the weight elsewhere. It argues that model quality, legal issues, data and risk are not what mainly holds AI back, and that the bigger problem is tools that do not learn and do not fit into the work. Its researchers wrote that most custom and vendor-sold tools fail "due to brittle workflows, lack of contextual learning, and misalignment with day-to-day operations." Put plainly, the tools broke easily in real use, did not learn from feedback or adapt to the situation, and did not fit how staff work from day to day.

What should you check before funding the next pilot?

Settle six things in writing before anyone signs, each aimed at one of the causes above.

Fig. 3

Six reasons AI pilots stall, and the check that answers each before you fund one

The causes named by Gartner, MIT NANDA and a survey of 43 health systems, each paired with a check to settle in writing

  1. Measure the task as it runs today. Write down how many staff hours it takes each week, how long it takes to turn around and how often it goes wrong, so the pilot has a number to beat when it ends.
  2. Test the tool on your own work. Use made-up records or records with patient details removed, and count the errors your staff would have to fix. One way to run that test is laid out in the post on testing an AI tool on your own tasks.
  3. Check the data the tool needs. List the systems it reads from, confirm the data there is complete and current, and name the person who fixes it when it is not.
  4. Name the people and the step where it fits. Say who will use it, at which step of their day and inside which system, and make one of those people the owner of the pilot.
  5. Price it at full use. Ask for the cost per user or per use at the volume you would run after the pilot, plus connecting it to your systems, support and the staff time to check its output. The post on AI model price cuts and vendor contracts lists the price terms to check before you sign or renew.
  6. Agree on the controls and the decision rule before go-live. Decide who reviews the output and what gets logged, then write down which result, by which date, means you expand the tool, extend the pilot or stop it.

Do these checks fit a medical practice or a health center?

Yes, though the numbers above come from large health systems and from research across all industries, so a practice or health center should read them as signs of where pilots go wrong rather than as its own odds. None of the six checks needs a data team. A practice can start with the first one and count the staff hours a task takes each week before it talks to a vendor, so any quote can be weighed against that number.

If you would like help with the checks, the AI Readiness Sprint takes two weeks and costs $9,500, fixed. It measures where your staff's hours go and ranks each AI use by the hours it would save against what it would cost to build and run. It also builds a working prototype of the top use on sample data or records with patient details removed, so your team can try it before you pay for a build. An organization that wants one person to own these checks across every AI pilot, without a full-time hire, can bring in a fractional chief data and analytics officer.

Related service

Related articles

More on AI adoption: 31 charts and 30 sourced facts.

Insights by email

Does this look like your organization?

Book a 20-minute call with Tyler to talk through your version of this problem and what fixing it would cost.

Prefer to skip the call? Get a written proposal