BlogAI adoption
Why healthcare AI pilots stall, and what to check first
Immature AI tools topped the list when 43 large US health systems were asked what was holding back their AI projects. In that survey, run in fall 2024 among nonprofit health systems and published in the Journal of the American Medical Informatics Association (JAMIA) in May 2025, 77% named immature AI tools as one of their two biggest barriers to developing or deploying AI, ahead of financial concerns (47%) and regulatory or compliance uncertainty (40%).
77%
of 43 US nonprofit health systems named immature AI tools one of their two biggest barriers to AI, fall 2024
In a fall 2024 survey of 43 US nonprofit health systems, published in JAMIA in May 2025, 77% named immature AI tools as the biggest or second-biggest barrier to developing or deploying AI.
Source: Poon and others, Adoption of artificial intelligence in healthcare: survey of health system priorities, successes, and challenges, Journal of the American Medical Informatics Association.
Research across all industries names other causes. Gartner predicted in July 2024 that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, and it named four causes: poor data quality, inadequate risk controls, escalating costs and unclear business value. Most of these causes can be checked before a pilot is funded, and the six checks near the end of this piece show how.
How many organizations get an AI tool into production?
Few, by the count in MIT NANDA's 2025 report on AI in business, which sets a high bar for production. Of the organizations in its research, 60% evaluated custom or vendor-sold generative AI tools, 20% reached a pilot and 5% reached production. The report covers healthcare among other industries and rests on interviews with 52 organizations, a survey of 153 senior leaders and a review of more than 300 public AI projects.
It counted a tool as in production once users or executives described a marked and lasting effect on productivity or profit. The report presents its results as preliminary findings and calls these figures directionally accurate, because they come from interviews rather than from company records, and it adds that success definitions may differ across organizations.
Fig. 1
Only 5% of organizations put a custom or vendor-sold generative AI tool into production
Share of organizations in MIT NANDA's 2025 research that evaluated, piloted or put into production a custom or vendor-sold generative AI tool
Show the numbers as a table
| Category | Value |
|---|---|
| Evaluated a tool | 60% |
| Reached a pilot | 20% |
| Reached production | 5% |
How far have health systems taken AI?
Few have taken it across the whole organization. The consulting firm Deloitte surveyed 180 top executives at large health systems in the United States and five other countries in August and September 2025, for its 2026 Global Health Care Outlook. About 30% of the health systems reported running generative AI at scale in select areas of the organization, and 2% reported deploying AI across the entire organization.
A year earlier, the JAMIA survey found a similar gap in the six uses of AI in its clinical documentation and chart review categories. Ambient notes, AI that turns the conversation between a clinician and a patient into a draft note, was furthest along: 6 of the 43 health systems (14%) had deployed it fully and 20 (47%) had deployed it in limited areas. The authors describe ambient notes as rapidly advancing, and every health system that answered had at least begun developing or piloting it, which no other use in the survey matched.
The other five uses were further behind. One health system had fully deployed AI for abstracting data for clinical registries, none had fully deployed any of the other four, and 57% to 74% of the health systems that answered were still developing or piloting each of the five. Other kinds of AI were further along: 48% of the health systems had fully deployed AI for early detection of sepsis, and 40% had fully deployed AI for imaging and radiology.
Fig. 2
Ambient notes had spread, but full deployment of AI for notes and charts was rare
Stage reached by US nonprofit health systems for the six AI uses in the survey's clinical documentation and chart review categories, share of the 41 to 43 that answered for each use, in percent, fall 2024. The rest reported no activity.
Show the numbers as a table
| Category | Deployed fully | Deploying in limited areas | Developing or piloting |
|---|---|---|---|
| Ambient notes | 14% | 47% | 40% |
| Abstracting data for registries | 2% | 10% | 57% |
| Inpatient chart summaries | 0% | 7% | 71% |
| Clinic chart summaries | 0% | 7% | 74% |
| Ambient nursing | 0% | 2% | 67% |
| Drafting care plan notes | 0% | 2% | 68% |
Why do AI pilots stall?
The sources point to the tools themselves and to how organizations prepare for them, but they weigh the causes differently. The health systems in the JAMIA survey put immature tools first, and the authors write that "fit-for-purpose and practical evaluations are necessary before an AI tool is deployed, during its implementation, and in an ongoing way," so that a tool's benefits keep justifying its costs and risks.
Gartner's four causes are as much about money and management as about technology. Its analyst Rita Sallam said that with generative AI, "costs aren't as predictable as other technologies." Deloitte writes that health systems are under increasing pressure to show a clear return on their AI. In its survey, 51% of respondents said they had not measured the returns or that it was too soon to see results, while 31% reported "moderate" financial returns and 3% "significant" ones.
MIT's report puts the weight elsewhere. It argues that model quality, legal issues, data and risk are not what mainly holds AI back, and that the bigger problem is tools that do not learn and do not fit into the work. Its researchers wrote that most custom and vendor-sold tools fail "due to brittle workflows, lack of contextual learning, and misalignment with day-to-day operations." Put plainly, the tools broke easily in real use, did not learn from feedback or adapt to the situation, and did not fit how staff work from day to day.
What should you check before funding the next pilot?
Settle six things in writing before anyone signs, each aimed at one of the causes above.
Fig. 3
Six reasons AI pilots stall, and the check that answers each before you fund one
The causes named by Gartner, MIT NANDA and a survey of 43 health systems, each paired with a check to settle in writing
- Measure the task as it runs today. Write down how many staff hours it takes each week, how long it takes to turn around and how often it goes wrong, so the pilot has a number to beat when it ends.
- Test the tool on your own work. Use made-up records or records with patient details removed, and count the errors your staff would have to fix. One way to run that test is laid out in the post on testing an AI tool on your own tasks.
- Check the data the tool needs. List the systems it reads from, confirm the data there is complete and current, and name the person who fixes it when it is not.
- Name the people and the step where it fits. Say who will use it, at which step of their day and inside which system, and make one of those people the owner of the pilot.
- Price it at full use. Ask for the cost per user or per use at the volume you would run after the pilot, plus connecting it to your systems, support and the staff time to check its output. The post on AI model price cuts and vendor contracts lists the price terms to check before you sign or renew.
- Agree on the controls and the decision rule before go-live. Decide who reviews the output and what gets logged, then write down which result, by which date, means you expand the tool, extend the pilot or stop it.
Do these checks fit a medical practice or a health center?
Yes, though the numbers above come from large health systems and from research across all industries, so a practice or health center should read them as signs of where pilots go wrong rather than as its own odds. None of the six checks needs a data team. A practice can start with the first one and count the staff hours a task takes each week before it talks to a vendor, so any quote can be weighed against that number.
If you would like help with the checks, the AI Readiness Sprint takes two weeks and costs $9,500, fixed. It measures where your staff's hours go and ranks each AI use by the hours it would save against what it would cost to build and run. It also builds a working prototype of the top use on sample data or records with patient details removed, so your team can try it before you pay for a build. An organization that wants one person to own these checks across every AI pilot, without a full-time hire, can bring in a fractional chief data and analytics officer.
Sources
- Poon and others, "Adoption of artificial intelligence in healthcare.
- Gartner, "Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025," press release, July 29, 2024.
- MIT NANDA, "The GenAI Divide.
- Deloitte, "2026 Global Health Care Outlook," December 11, 2025.
Related service
Related articles
- In my map of 44 healthcare problems, 30 need no patient dataOctober 9, 2026, 3-minute read
- Building AI so each project makes the next one cheaperSeptember 22, 2026, 4-minute read
More on AI adoption: 31 charts and 30 sourced facts.
Insights by email
Does this look like your organization?
Book a 20-minute call with Tyler to talk through your version of this problem and what fixing it would cost.
Prefer to skip the call? Get a written proposal