ServicesAI adoption

AI Vendor Evaluation

Two weeks, from $7,500, fixed. Before you sign, each vendor on your shortlist is tested on cases from your own organization, puts in writing how it would handle your data, and has its full three-year cost added up. You get a recommendation with the reasoning shown: buy one, build instead, or wait.

Prefer to skip the call? Get a written proposal

Run by Tyler Kent, who has designed and shipped 10+ AI products to production. His background

Built for

  • Organizations comparing AI vendors with nobody assigned to check their claims
  • Hospitals and medical groups choosing ambient documentation, prior authorization, patient messaging or revenue-cycle AI
  • Digital health and payer teams choosing a model provider or an AI platform
  • Teams whose counsel asked what happens to their data and got no clear answer

Why it's worth it

Today

Each vendor demos the cases it handles best. Nobody reads the contract terms about your data until after the decision, and nobody adds up the full cost, including integration and staff time, before signing.

After

You see every vendor work through the same cases from your organization side by side, with its answers about your data and its three-year cost in writing. You choose with a memo your board can read and a list of contract terms to ask for.

What you get

  1. Week 1

    Requirements matrix

    A one-page list of what the tool must do, drawn from your workflow and two or three staff interviews, each line ranked must, should or nice to have. It's written before any vendor is scored.

  2. Weeks 1–2

    Scored comparison

    Each vendor runs the same de-identified cases from your organization (patient details removed) and is scored on them, along with its security and HIPAA posture, the work to connect it to your systems, and its total cost over three years.

  3. Week 2

    Recommendation memo

    Buy a named vendor, build instead, or wait, with the build-versus-buy math in dollars. It comes as an editable document in your template, ready for your board or a second opinion.

  4. Readout

    Negotiation points

    The contract terms to ask for before signing: limits on how your data is used, no training on your data, exit and export rights, and service levels you can enforce, meaning the uptime and response times the vendor commits to.

How each evaluation is run

Each evaluation rests on a requirements matrix written before any demo and on tests with your own cases.

Want to see the method first? Read how to score an AI vendor before the demo, or the healthcare AI evidence map.

Tyler Kent

Who runs it

Tyler Kent

Tyler has designed and shipped 10+ AI products to production, and he built the four on the Portfolio page himself.

He also reviews research submitted to AMIA, the American Medical Informatics Association, and has published nine journal papers. Each evaluation asks of a vendor's evidence what a reviewer asks of a study: who was studied, and what the tool was compared with.

25+

courses on AI adoption taught to leadership teams and staff

10+

AI products designed and shipped to production

13+

years in data science and analytics, most of it in healthcare

9

peer-reviewed journal papers, including JAMA Network Open

Choose your version

Each version has one fixed price, written into your proposal. Nothing is billed by the hour.

Up to three vendors

For a shortlist you have already narrowed

Price: $7,500

One time

2 weeks

Every vendor runs the full test set and is scored on each requirement.

What's included in Up to three vendors
  • A requirements matrix written before any demo
  • Tests on your own de-identified cases
  • Written answers on the BAA, which other companies the vendor sends your data to, how long it keeps the data, whether it trains its AI on your data, and how you get your data back if you leave
  • Three-year cost and the build-versus-buy math
  • A recommendation memo and the contract terms to ask for
  • A readout with your team

Four or five vendors

For a crowded market

Price: $9,500

One time

2 weeks

The same tests for every vendor. A longer list is cut to five in week one.

What's included in Four or five vendors
  • Everything in Up to three vendors, for four or five
  • A longer list cut to five in week one, against the requirements matrix
Payment and terms
  • Half at signing and half at delivery
  • Paying by purchase order? There's no deposit: we invoice half at kickoff and half at acceptance
  • No software sold and no referral fees
  • No patient data needed

Start and finish dates are set in writing in your proposal.

Questions

Do you resell or get referral fees from any AI vendor?

No. Kent Applied sells no software and takes no referral fees, so the recommendation can be any vendor, none of them, or a build.

How many vendors can be on the shortlist?

Up to five. Three or fewer is $7,500, and four or five is $9,500. A longer list is cut to five in week one, against the requirements matrix.

Do you need our patient data to test the vendors?

No. Capability tests run on de-identified or synthetic cases you approve. Real patient data never goes to the vendors. We check how each would handle it by reviewing its documents: its BAA, its subprocessors (the other companies it sends data to), how long it keeps data, and whether your data trains its models.

How do you test an AI scribe before we buy one?

With scripted visits. We write short visit scripts with fictional patients, your clinicians and staff act them out, and every scribe on your shortlist drafts a note from the same visits. The scripts include the situations where published studies found scribes make mistakes. In an audit of three commercial scribes, posted as a preprint that has not been peer reviewed, about one note in three had a verified error, most often in allergy and medication details. In a randomized trial at UCLA, the physicians who reported inaccuracies most often described details left out of the note. In a study of acted-out visits with an interpreter, the two scribes tested carried 55% and 60% of the interpreter's mistakes into the note. In a preprint covering 54,160 visits at one safety-net health system, notes from English visits were signed with at least 80% of the scribe's words left unchanged in 57.6% of visits, against 48.7% to 51.1% for visits in other languages. A 2025 report from the Peterson Health Technology Institute says some scribes have trouble summarizing a visit with a patient and several caregivers, because they can't always tell the voices apart. So the scripts cover medication and allergy changes, an interpreter visit, a visit in another language, a family in the room and visits from your own specialties. Your clinicians score each note without knowing which scribe wrote it: what it got wrong, what it left out, whether the medications and allergies are right, whether it fits your specialty's format, and how long it took to fix before signing.

Why not test the scribes on recordings of our real visits?

Because a recording of a real visit is patient data. HIPAA's de-identification standard lists voice prints among the identifiers that must be removed, so we treat visit audio as patient data. Scripted visits give every scribe the same test with no real patient in it, and nothing about a real patient goes to the vendors or to Kent Applied. Your clinicians sign in to the scribes during the test; we never do.

What if none of the vendors is good enough?

Then the memo says so. No purchase, a build, or waiting for software you already own to add the feature are all valid results.

What if the right answer is to build instead of buy?

The memo shows the cost of building next to the three-year cost of buying. Kent Applied can do the build as a fixed-price Custom AI Tool, or you can take the memo to another team.

Can you join the vendor negotiation?

Yes. The memo lists the contract terms to ask for, and Tyler can join the negotiation calls. That's priced as a fixed add-on in your proposal.

Book a 20-minute call

Tell us which vendors you're considering and when you need to decide. If we can help, you'll get a fixed-price proposal within 48 hours.

Prefer to skip the call? Get a written proposal

Or email tyler@kentapplied.com, or connect on LinkedIn.