News: Analysis compares sepsis early warning scores with and without AI

Additional transparency and oversight of early warning scores (EWS) for sepsis may be warranted, according to a recent study published in JAMA Network Open. Of the 30 EWSs available to healthcare organizations, the study compared six with a variety of complexities. They found eCART, a model constructed using machine learning, outperformed the others, Medscape Medical News reported.

Researchers measured the receiver operating characteristics curve (AUROC) of each as well as the positive predictive values at both the moderate- and high-risk matched thresholds for sepsis across 362,926 patient encounters. The primary outcome variable was clinical deterioration.

Three of the scores used models derived from artificial intelligence (AI), and the other three did not. The eCART model scored highest, though they found the simple National Early Warning Score and NEWS2 scores were as good or better than the other two AI models studied. The Epic Deterioration Index was one of the worst, with a low AUROC score and the lowest positive predictive values.

“This cautionary tale shows that EWSs based on statistically sophisticated modeling can have subpar performance,” Medscape Medical News reported. “The old maxim that ‘all models are flawed but some are useful’ holds true in the era of AI.”

Medscape Medical News also pointed out that the study authors developed the eCART and hold patents for its use, indicating a potential conflict of interest.

Editor’s note: To read Medscape Medical News’ coverage of this story, click here. To read the JAMA Network Open study, click here.