Kaiser Nurses Warn AI Monitoring Could Harm Patient Care

Kaiser Permanente nurses who answer advice and triage calls say their duty of care for patients is being increasingly threatened by workplace surveillance.

Seven current and former nurses told The Markup that those who spend more than 15 minutes on a call with a patient routinely face criticism from Kaiser management or get called into performance evaluation meetings. Call time, they said, factors into monthly performance scores they receive.

In addition to tracking call length, they said Kaiser uses software that tries to predict on a daily basis whether they’re being unproductive or failing to answer calls quickly. Artificial intelligence systems have also been used to rate their empathy and tone of voice.

AI Surveillance Sparks Labor Disputes at Kaiser Permanente

Their comments come as the California Nurses Association begins contract negotiations with Kaiser this month. AI is likely to be a key issue. Kaiser nurses went on strike against AI for one day in March and picketed against AI last fall. The CNA is bargaining for 25,000 nurses, including 1,000 in call centers.

At the same time, California lawmakers are also considering bills to regulate AI in the workplace. One proposal would protect doctors and nurses from retaliation when they override automated care recommendations.

Kaiser defended its use of AI, saying it deploys the technology with human oversight and with patient safety in mind. It said it does not use “average handle time” to assess performance.

Kaiser Permanente is the largest private employer in California, providing healthcare services to more than 9 million people in the state and to 3 million other Americans. Kaiser’s use of AI could set an important precedent for managing workers. It could also affect patient care. The company’s approach may show how healthcare providers balance cost-cutting automation with human care.

Raquel Alvarez Sanchez, a Kaiser Permanente advice nurse in Vallejo since 2010, said she was on a call with a patient who was suicidal last year that took more than an hour because she had to wait for police to arrive before hanging up. She tried to make the man feel cared for, even though she was cognizant that staying on the call that long would throw off her average call time for weeks and could lead to questions from management. Sanchez, a union steward, said she’s accompanied colleagues to performance evaluation meetings, where they were found to have done everything right on a call — except staying on the line for more than 15 minutes. She said she hasn’t seen nurses get fired for doing that, but she fears that continued pressure can lead nurses to quit or retire early.

AI Surveillance Raises Concerns About Compassionate Patient Care

“I think at some point all of the nurses have been talked to about their average handle time,” she said. “The only thing I can think of is they’re doing it for profit.”

Another nurse spoke with The Markup anonymously because she feared retaliation. She described how surveillance affected a patient call last year. At first, the nurse thought her patient was suicidal. The older woman had just received a terminal cancer diagnosis. The nurse soon realized she was in shock and needed someone to talk to.

The nurse wanted to comfort the woman and show compassion. The patient also cared for her daughter. But the nurse feared that a longer call would hurt her monthly performance score and lead to a reprimand. She became a nurse to provide people with compassionate care, but “I had to ask myself: Am I going to get disciplined for going off script or saying more than what is necessary?”

Kaiser Permanente says its performance evaluations help improve patient outcomes. A company spokesperson said, “Kaiser Permanente does not use Average Handle Time to assess agent performance or enforce call time metrics. Any tools used in contact center settings support our quality assurance efforts and have human review and oversight. ”In a statement to The Markup and CalMatters, spokesperson Vincent Staupe said Kaiser uses AI responsibly and maintains human oversight. He said the company prioritizes patient safety, privacy and equity. However, he added that, “As a large organization, we do not share specific information about internal technology systems for security and operational reasons.”

Related Post: Nigerian-Born PhD Student Wins $30K Google Contest With AI Tool for Early Disease Detection

Could AI Monitoring Put Patient Safety at Risk?

The effect of algorithmic management on patient care remains unclear. Researchers also do not know how limiting triage and advice calls affects patients. Kaiser nurses cannot confirm whether these pressures lead to harmful outcomes. Their contact with patients ends when they hang up the phone. A 2024 public records request by CalMatters found no patient complaints about Kaiser call times in records from the California Department of Managed Health Care. Nurses, however, insist that the risks to patient safety and care are real.

Michele Ramos, a patient advocate at Consumer Watchdog, said many Kaiser patients begin their care through the advice line. They later complain to her, mostly about issues at Kaiser facilities.

Ramos said time pressures may reflect a broader pattern of putting costs ahead of quality. Kaiser paid a record $50 million fine in a settlement over behavioral health care delays. Investigators found that the company delayed appointments beyond legal limits. They also found that Kaiser too often placed patients in group therapy instead of individual sessions. Kaiser also settled with the U.S. Department of Labor after investigations into its substance use and mental health services. In 2002, Kaiser faced criticism for paying non-nurse call center workers bonuses for keeping calls short. Nurses who spoke with The Markup and CalMatters said they have not encountered that practice today.

Strict Call-Time Limits Raise Patient Safety Concerns as AI Surveillance Looms

“Kaiser’s been known through the years to manage dollars over managing care, and I think this would be a contributor to that, which is only going to fail patients,” Ramos added.

Nurses said managers pressure them to keep calls under 15 minutes, even when patients need more time. These calls may involve multiple symptoms, chronic illnesses or questions from new parents. Other patients need extended health education or reassurance after life-changing news. Calls involving interpreters often take 30 minutes or longer, nurses said. About four in 10 Californians speak a language other than English. Half of those residents do not speak English well, according to a state environmental health agency.

“The amount of time that Kaiser is giving us to complete a call is sometimes not safe,” said one nurse, who asked to remain anonymous due to fear of retaliation.

“People can get hurt,” said Charlotte Capulong, who has worked in nurse call centers for 22 years and helped organize Kaiser nurses against the AI tone-of-voice tool. Capulong said nurses felt harassed by managers in meetings she attended as a union rep, even if they successfully carried out all other duties of their jobs except completing calls within 15 minutes.

Nurses Push Back Against AI Surveillance

“You aren’t calling Comcast. We’re dealing with life here,” she said.

Capulong and other nurses said they must follow scripts and offer no more than two or three pieces of advice per call. Nurses may have to withhold guidance or risk a performance review.

The nurses say artificial intelligence could make the surveillance nurses encounter on the job worse.

Nurses told The Markup and CalMatters that Kaiser began testing an AI tool in summer 2024. The tool attempted to assess empathy and tone in nurses’ and patients’ voices. Nurses responded by circulating a petition. They called for patient privacy, greater transparency and the right to exercise professional judgment. They also urged management to seek nurses’ input and feedback. The signature campaign used the same tag line that nurses used at protests outside San Francisco hospitals earlier that year: “Trust nurses, not AI. The AI tests ended in November 2024, but union representatives were told that managers may bring the program back in the future.

Nurses said they already felt harassed by workplace surveillance. The planned AI grading system made those concerns worse, Sanchez said.

AI Grading and Call Pressure Raise Patient Safety Risks

Another nurse speaking on condition of anonymity said “AI did not understand our job and would grade us wrong all the time.”

A Kaiser spokesperson declined to respond to questions about the AI tool or answer questions about the use of AI and other automated systems in the company’s call centers and healthcare facilities, including for evaluating nurse performance or whether patients were informed about the use of AI to evaluate their empathy and tone.

Nurses said they have little time between calls, even after speaking with suicidal patients or people experiencing mental health crises. Some patients are near death.In the past, nurses had about 10 minutes to finish patient notes or recover after a difficult call. Today, they say they often get 30 seconds or less when lines are busy. They may get more time during quiet periods or with a manager’s permission.

Nurses warned that this pace can lead to mistakes, including missed signs of a patient’s declining well-being.

How surveillance and AI shape nursing

Critics say excessive monitoring can lower morale. Workers may feel less trusted and lose control over their jobs. They can end up acting like algorithm monitors rather than skilled professionals. UC Berkeley Labor Center Technology and Work Program director Annette Bernhardt has warned that algorithmic management can turn people into fleshy robots, echoing complaints from an Amazon factory worker who The Markup and CalMatters interviewed last year. A 2023 academic survey examined call centers in four developed countries. It found that AI monitoring left workers with less time between calls. Workers were also more likely to feel emotionally drained. Nearly half said AI tools made their jobs more stressful.An earlier study by the same researchers, Virginia Dolleghast of Cornell University and Sean O’Brady of McMaster University, found that performance monitoring increased emotional exhaustion

Dolleghast said Kaiser nurses’ experiences reflect a wider trend. Across industries, persistent surveillance increases stress for workers handling complex and emotionally charged issues: Across different industries, persistent surveillance is increasing stress levels for workers who are resolving complex, emotionally-charged issues.

“Stress and burnout can lead to more mistakes across a range of areas, and in the healthcare setting that is much higher risk because you’re dealing with people’s lives and their health,” she said.

AI Performance Metrics Can Override Nurses’ Clinical Judgment

The converse can be true: Workers who are given more discretion to decide the pace and timing of their work experience higher levels of job satisfaction and less absenteeism.

Nurses nationwide are encountering AI and similar workplace software more often. In a 2024 National Nurses United survey, half of more than 2,000 respondents said their employers use algorithms to analyze health records.These systems may assess a patient’s frailty or predict how many hours of care they need. Two-thirds of respondents said their own assessments had conflicted with computer-generated predictions. Six in 10 said they did not trust their employers to prioritize patient safety when using AI.

Pa Vue has worked in nurse call centers for most of the past decade. She said Kaiser nurses regularly discuss call efficiency with managers. They also receive monthly performance scores. Vue recalled losing points after repeating advice to a patient whose symptoms raised concerns about possible heart problems.

Related Post: Gates Foundation and OpenAI Launch $50M AI Health Initiative in Africa

Unions Push for AI Safeguards in Healthcare

As a union representative at performance meetings, Vue has seen managers question calls they consider too long. She has also seen nurses receive lower scores for rejecting software recommendations based on their clinical judgment. Some nurses also lose points for scheduling appointments without consulting a doctor.

She believes that efficiency aims accelerated by technology can hinder a nurse’s ability to focus and reduce the quality of care that patients pay for.

“I’m not against the use of AI as long as it’s beneficial to the patient but in this particular use [empathy and tone monitoring] it’s to increase productivity and improve efficiency and cut costs. Kaiser is forgetting we aren’t just a call center for customer support, we’re nurses, and we’re there to take care of patients,” she said.

As AI advances and businesses push workers to use it, unions are demanding safeguards in contract negotiations. Managers use surveillance technology across many industries. They may use it to improve safety, increase profits or train AI to perform workers’ jobs.

Kaiser therapists worry that the company could use therapy transcripts to train AI models or replace their jobs. National Union of Healthcare Workers spokesperson Matt Artz told The Markup that negotiations are ongoing.

How Kaiser uses AI

Kaiser uses AI in several areas beyond nurse call centers. One system identifies hospitalized patients at risk of adverse events by analyzing electronic health records. Another, called Preventus, helps determine when to discharge patients. Doctors and therapists use Abridge to record visits and convert speech to text. This lets them document conversations without taking notes. Nurses have also encountered AI-powered remote monitoring for patients who need extra care at Kaiser facilities in the Bay Area.

Cathy Kennedy, president of National Nurses United and the CNA, sees AI empathy detection as part of a broader effort to limit nurses’ autonomy. She believes Kaiser uses the technology to increase efficiency. Kennedy says developers should prove AI systems are effective and equitable before deployment.

Kaiser’s contract requires the company to notify the nurses’ union about new technology. However, the CNA says nurses sometimes receive no notice. Union leaders are tracking AI systems across Kaiser and asking members to report new deployments. This effort helps the CNA challenge new technology, as it did with the empathy-monitoring trial. The union also stopped a pilot program that would have replaced bedside nurses caring for confused patients with cameras.

Related Post: Black-Led Tech Firm Calendly Valued At $3 Billion After $350M Series B Raise

California Lawmakers Push for AI Workplace Protections

Debru Carthan, a Kaiser radiologist, is involved in workplace disputes over AI. She belongs to the Service Employees International Union and the Coalition of Kaiser Permanente Unions. She also serves on a committee that discusses AI and emerging technology at Kaiser. The coalition encourages frontline workers to report new AI deployments. It says management often introduces these systems without notice or worker input. Carthan worries that AI tone detectors could discriminate against nurses from different cultures. She also believes healthcare companies often prioritize profits over patient care.

California lawmakers have responded to worker AI concerns both inside and outside the healthcare sector. They tried and failed last year to address how AI impacts workers like call center nurses. Assembly Bill 1018 and Senate Bill 7, two bills endorsed by the CNA, would have required employers to inform workers before using automated systems on the job to do things like promote or discipline workers or evaluate job performance, but Gov. Gavin Newsom vetoed SB 7, and, facing strong opposition from companies including Kaiser Permanente, AB 1018 failed to pass for the third consecutive year.

AI Monitoring Raises Fears About Losing the Human Touch in Nursing

Earlier this year, lawmakers reintroduced a new version of Senate Bill 7, now called Senate Bill 947. Another bill would prohibit employers using AI to predict the emotional state of their employees. Yet another bill would protect doctors and nurses from retaliation if they override recommendations generated by an automated system and require healthcare providers to supply employees with an inventory of automated systems once a year. Kaiser declined to share a list of AI systems in use when asked by The Markup and CalMatters.

Altogether, CNA and the affiliated California Labor Federation support roughly half a dozen bills to regulate the use of AI in the workplace. Calling AI a central issue in the next presidential election, members of the California Labor Federation and labor leaders from Democratic primary states held a press conference in Sacramento earlier this year to say that if Newsom wants to become president then he needs to pass laws protecting workers from AI. “It’s an ongoing fight, and it’s a fight well worth having,” Kennedy said. “Whenever there are other unions in discussion about artificial intelligence we are in solidarity with them.”

The nurse that withheld compassion to a terminal cancer patient she thought was suicidal said she believes monitoring and scoring systems turn nurses into automatons that check boxes.

“I used to use humor as a way to help patients heal, and I don’t feel comfortable doing that here because I know the calls are being recorded. You can always tell when a patient appreciates the humor or your personal compassion, but I don’t feel like call centers have tolerance for that because that’s not part of the script,” she said. “That really takes away from the whole point of being a nurse and what patients come to know from nurses.”

Written by: Khari Johnson for The Markup

This story was reported with contributions from Lam Thuy Vo and Ana Ibarra.

This story was produced by The Markup and reviewed and distributed by Stacker.