ExoBrain
AI safetyenterprise AIgovernance and regulationhealthcare and biotech

AI on the frontlines of healthcare

While AI-driven diagnostics and drug discovery show significant promise, the UnitedHealth controversy highlights critical risks regarding accountability and the need for human-centred oversight in medical AI deployment.

Joost de Jonge

Joost de Jonge

2 min read
AI on the frontlines of healthcare

UnitedHealth, and the entire health insurance industry, has been under intense scrutiny following the tragic shooting of CEO Brian Thompson in New York. Police are investigating potential connections between the shooting and healthcare decisions, which has also brought attention to the company’s AI practices. For some years UnitedHealth has faced questions about its system, nH Predict, which is accused of systematically denying claims to elderly patients.

A Senate subcommittee report shows that denial rates for post-acute care increased from 10.9% in 2020 to 22.7% in 2022. A class action lawsuit filed in November 2023 claims the AI system has a 90% error rate in its decisions. The system evaluates patients based on diagnosis, age, and physical capabilities, but provides generic recommendations that often fail to consider individual circumstances.

This tragic episode highlights the need for more thorough design and implementation, transparency, accountability, and a human-centred approach in deploying AI for medical decision-making. Despite these controversies, 2024 has seen AI delivering significant contributions to the industry.

One notable example is AI’s role in advancing cancer diagnostics. The latest systems have shown exceptional accuracy in identifying tumours from imaging scans, often outperforming human radiologists in speed and precision. Recent studies have demonstrated AI tools detecting early-stage breast cancer with high accuracy, helping to ease the burden on overworked healthcare professionals.

In biotech and drug discovery, companies like DeepMind have used AI to predict new structures, speeding up the identification of therapeutic targets. This year’s breakthroughs include several AI-designed drugs entering clinical trials, with some showing promise for rare diseases previously considered untreatable.

Takeaways: The UnitedHealth case serves as a cautionary tale about the risks of deploying AI without sufficient oversight. As we look to 2025, several trends are set to define AI’s role in healthcare. Biotech startups are experiencing a surge in funding, with AI-acceleration at the forefront. We can expect further progress in precision medicine, where AI tailors treatments to an individual’s genetic profile, lifestyle, and environment. These advanced systems have the potential to rebuild patient trust, improve access to care, enhance treatment accuracy, and reduce costs across the board.