12 October 2026 · Medows
Verify Buttons Don't Build Trust in Health AI
KFF's new Monitor finds only about half of users click a chatbot's verify option, and it barely moved trust. What that means for clinical AI design.
Picture the ward at 7 a.m. A tool offers a confident answer, with a small "verify" link beside it. How many doctors click it? And if they don't, what was the link for?
A KFF Monitor issue published on October 8, 2026 puts numbers on a version of that question. It is worth reading for doctors, not just policy people.
The pilots are getting more autonomous
KFF's piece opens with the federal push toward AI that does more than assist. It describes the FDA's TEMPO digital health devices pilot, which tests tools that could interact with patients, provide therapy, diagnose, or recommend and adjust treatment. Per KFF (citing The New York Times), more than 200 companies have received CMS approval to test such tools, and the FDA has allowed four of them for use with Medicare patients outside its standard approval process.
KFF names two examples: a chatbot that delivers cognitive behavioral therapy, and a federally funded project to build AI that would adjust medication for heart failure patients, under legally required physician oversight.
We covered the pilot's regulatory shape earlier. The new part is the evidence KFF gathers about how people actually respond to AI output.
What the trust research says
KFF summarizes a chatbot experiment published in the Journal of Computer-Mediated Communication. A more conversational chatbot earned more trust, partly because people saw it as less rigid and more capable of judgment. That held even for false claims. One example was the claim that ginger works better than chemotherapy for cancer.
Then the finding that matters for anyone building clinical tools. Offering a verification option did not produce a statistically significant change in trust. Only the people who clicked to check showed lower trust, and only about half of those offered the option clicked.
Read that again. Half did not check. The verify button existed, and it changed almost nothing in aggregate.
KFF also reports that a PNAS study of 8.7 million tweets found mentions of scientific authority were linked to greater willingness to share content, including false content. Authority cues travel well. Accuracy travels less well.
The recall record
KFF also cites an August 2025 JAMA Health Forum research letter that examined 950 AI devices the FDA had cleared as of late 2024. Sixty devices were recalled a combined 182 times. The recalls were mostly for diagnostic or measurement errors, and nearly half came within the device's first year on the market.
KFF adds that the FDA has said newer generative AI uses may be harder to evaluate, because their outputs can be unpredictable and can change over time. The agency has not yet set rules for the category, and a public comment period runs through October 19, 2026.
What this means on the ward
None of this is clinical guidance. It is a design observation.
If half of users never click "verify," then verification cannot live behind a click. It has to be the default shape of the answer. The source sits next to the claim. The patient's own chart data sits next to the suggestion. The uncertainty is visible without asking.
A tired registrar on a 28-patient list does not have a spare minute for optional diligence. A tool that makes checking a separate step has quietly decided that most checks will not happen.
This is the part of our thesis we hold most firmly. A doctor's AI should show its work in the same view as its answer, tied to the record in front of them, so that trust is earned by what is on the screen and not by how fluent the reply sounds. Fluency is cheap. KFF's summary suggests it is also persuasive, which is exactly the problem.
The other half of the story is for the people writing the rules. Autonomous pilots will be judged on outcomes. Doctors will judge them first on whether they can see why.
Sources
- KFF Health Information and Trust, "AI in Health Care Moves Toward More Autonomous Roles," The Monitor, October 8, 2026: https://kff.org/health-information-trust/ai-in-health-care-moves-toward-more-autonomous-roles
Medows is a clinical AI workspace for the doctor on rounds. Learn more or write to alapan@medows.ai / alapanx@gmail.com.