Empirical Stress-Testing of Early Anastomotic Leak Prediction Under Label Noise, Selection Bias, and Threshold Dependence

Abstract

Postoperative prediction models are often judged by a narrow set of performance numbers, yet those numbers are produced inside a data system with limited event counts, incomplete measurements, and outcomes that are not directly observed but inferred from coded and procedural traces. Anastomotic leak prediction is a clear example. The event is uncommon, the first postoperative day contains only partial evidence, and the label used for training usually combines diagnosis codes with downstream clinical actions. Under these conditions, the practical meaning of a model depends not only on ranking ability, but also on how sensitive the reported results are to outcome definition, threshold placement, and the hospital’s own pattern of observation. This paper presents an empirical re-analysis of an early leak prediction experiment using structured perioperative records. The analysis reconstructs model operating points from the published held-out metrics and then subjects them to three stress tests. The first examines how modest phenotype misclassification changes estimated prevalence, precision, and recall. The second studies the effect of observation-dependent variables such as intensive care placement and selectively measured laboratories on the stability of feature importance and thresholded decisions. The third examines how prevalence shifts across hospitals alter the positive predictive value of the same score, even when ranking quality is unchanged. Across these analyses, the main result is that the stronger nonlinear models remain informative, but their reported performance should be read as conditional on a specific label construction and a specific monitoring environment. The study therefore argues for empirical reporting practices that make those conditions explicit rather than presenting thresholded performance as if it were transportable without adjustment.