“
Another dire example involves an algorithm called “nH Predict”, used by UnitedHealth Group (the largest health care insurer in the U.S.) to determine the length of stays it would approve for patients in nursing homes and care facilities. In a class-action lawsuit61 filed in November 2023, the estates of the two named plaintiffs—deceased at the time of filing—alleged that UnitedHealth kicked them out of care too early, based on nH Predict’s output, even as the company knew the system had an error rate of 90 percent. The court filing says that UnitedHealth used this system anyway, counting on the fact that only a tiny group of policyholders appeal such denials, and that the insurer “[banked] on the [elderly] patients’ impaired conditions, lack of knowledge, and lack of resources to appeal the erroneous AI-powered decisions.” The families of the two plaintiffs spent tens of thousands of dollars paying for care that went uncovered by the insurer. Reporting from Stat News largely confirms62 the allegations in the lawsuit, namely that after acute health incidents, UnitedHealth aimed at getting elderly patients out of nursing homes and hospitals as fast as possible, even against the advice of their doctors. Moreover, when patients challenged denials, physician medical reviewers were advised by case managers not to add more than 1 percent of the prior advised nursing home stay. And case managers themselves were fired if they strayed from those targets.
”
”
Emily M. Bender (The AI Con: How to Fight Big Tech's Hype and Create the Future We Want)