When Hospital AI Overrules the Nurse

A hundred nurses outside Palo Alto

On August 27, more than 100 registered nurses in red union shirts blocked traffic outside Palantir's former Palo Alto headquarters. They were not protesting a data breach or a hack. They were protesting a scheduling algorithm.

National Nurses United, which represents more than 225,000 members, organized simultaneous protests in Los Angeles, Chicago, Washington D.C., and four other cities. The union called it the largest demonstration yet against the company's expansion into healthcare.

What the algorithm is actually doing

The complaints were specific, not abstract. HCA Healthcare, which operates roughly 190 hospitals across the US and UK, uses Palantir's tools for nurse staffing and scheduling. Mount Sinai Health System uses Palantir products to evaluate patient records and determine eligibility for at-home care.

Craig Cedotal, a pediatric nurse at Kaiser Oakland, put it in one line: "A machine should not make decisions about how we take care of each other." His ask was not to ban the software. It was for "more transparency and a greater say in the use of AI in hospitals."

That distinction matters. Nobody at the protest argued that software has no place on a ward. They argued that a staffing tool deciding who covers a shift, or a records tool deciding who qualifies for home care, is making a clinical judgment call without a name attached to it.

The bill trying to catch up

California's AB 1979 has been moving through the legislature at the same time. As of its August 20 amendment, the bill text requires health facilities to "take reasonable steps to ensure that a licensed health care professional, acting within their scope of practice, retains the ability to exercise independent professional judgment" whenever care is informed by an AI clinical decision support system. It also bars AI from independently performing, or directing unlicensed staff to perform, any clinical function that by law requires a professional license. Routine documentation and communication tools, like appointment reminders, are carved out.

According to KQED's reporting, lawmakers passed the bill on August 28 and sent it to Governor Newsom's desk.

Read together, the protest and the bill describe the same gap. Hospitals bought software to solve a documentation and operations problem, and some of that software quietly ended up making clinical calls nobody voted to hand over.

Why this is not just a labor dispute

It is tempting to read this as a familiar story: union versus vendor, staffing versus software. But the underlying complaint is the one every clinician eventually has with a black box tool, whatever the specialty. If a nurse cannot see why the schedule assigned her to a unit she has never worked, or why an algorithm flagged a patient ineligible for home care, she cannot push back on it the way she could push back on a charge nurse's decision. The reasoning is not on the chart. There is no note to challenge.

That is the design question underneath the protest signs: does the clinician using the tool see the "why," and does the tool leave the final call with them? A staffing or eligibility system that outputs a decision with no visible reasoning fails that test even when it is accurate on average. A system that shows its work, and treats a clinician's override as the expected path rather than an edge case, does not.

Ambient scribes and note-drafting tools mostly clear that bar already, since a clinician reads the draft before it becomes the record. Staffing and eligibility algorithms, as described in this reporting, do not obviously clear it, since their output can become the operational decision before anyone with a license has seen the reasoning behind it. That is the line worth watching as more of this software moves from the back office into decisions that touch a specific patient or a specific shift.

Sources

• Bay Area Nurses Protest Use of Palantir's AI Tech at Hospitals (KQED)
• AB-1979 Bill Text (California Legislative Information)
• AB-1979 Bill Status (California Legislative Information)