Alert-Budgeted Multimodal Risk Calibration for ICU Deterioration Forecasting from Asynchronous Physiologic and Radiologic Signals

Abstract

Intensive care prediction systems are usually judged by discrimination metrics, yet bedside deployment depends just as much on calibration, review burden, and the way heterogeneous data streams are synchronized. This is particularly true when structured physiologic measurements arrive hourly while narrative documentation is sparse, delayed, and semantically dense. In such settings, a useful model must do more than separate positive and negative cases; it must transform multimodal evidence into a stable risk signal that remains informative under low prevalence and finite alert capacity. This paper studies ICU deterioration forecasting from that operational perspective. A multimodal architecture integrates 48-hour windows of vital signs and laboratory values with contextual embeddings from radiology reports to predict death in the ICU, vasopressor initiation, or invasive mechanical ventilation within the next 24 hours. The cohort contains 74,822 ICU stays and 5,720,904 hourly samples, allowing analysis not only of ranking performance but also of alert density, work-up burden, and calibration recovery. On a held-out test set of 823,641 samples, the multimodal system reaches an AUROC of 0.7857 and an AUPRC of 0.1908, surpassing structured-only deep learning and classical baselines. At the reported operating point, the model generates approximately 30 alerts per 1,000 windows, yielding about 7.5 true deteriorations per 1,000 windows and requiring roughly four reviews per true event identified. Post-hoc temperature scaling reduces expected calibration error from 0.2129 to 0.0240 while preserving discrimination. The analysis indicates that multimodal context is most valuable not as an independent detector but as a mechanism for sharpening risk concentration and making thresholded deployment more efficient under class imbalance.