3 Key Takeaways
Plain English Summary- 01Core Finding
In this multicentre retrospective cohort study, we included data from 15 Danish hospitals (ten hospital networks) across the Capital and Zealand regions of Denmark, between Oct 16, 2018, and Oct 16, 2023.
- 02Study Evidence
Tested in a ehr cohort study + clinical ai evaluation (000 patients) at the real-world cohort stage.
- 03Practical Meaning
AI model accuracy depends on training dataset diversity and requires prospective validation across independent hospitals before routine rollout.
What This Means for You
Model complexity produced small average clinical utility gains, concentrated among patients initially classified as high risk by NEWS. These findings support prospective trials of risk-stratified monitoring based on the first NEWS assessment, testing simplified approaches for lower-risk patients while retaining full NEWS or electronic health record models for higher-risk patients.
The Discovery
The National Early Warning Score (NEWS) is widely used to predict patient deterioration, but retrospective validations face intervention bias and emphasise discrimination performance over clinical utility. We aimed to compare the clinical net benefit of NEWS against simplified scoring rules and machine learning models, to determine whether simpler approaches suffice or if complex models offer meaningful advantages, and to evaluate performance heterogeneity across patient subgroups. In this multicentre retrospective cohort study, we included data from 15 Danish hospitals (ten hospital networks) across the Capital and Zealand regions of Denmark, between Oct 16, 2018, and Oct 16, 2023. We included all adults (aged ≥18 years) with a hospital encounter with at least one NEWS assessment. Demographic and clinical data, NEWS data, and time of death were extracted from hospital and regional electronic health records. Patients were followed up until death or completion of the 24 h observation period after the initial NEWS score was recorded, whichever occurred first. The outcome was all-cause mortality within 24 h of the initial NEWS recording. We compared NEWS against both simpler and more complex approaches for predicting 24 h mortality: simplified NEWS (NEWS without blood pressure and temperature), Demographic Early Warning Score (DEWS; simplified NEWS with age and sex), and a model based on Extreme Gradient Boosting Early Warning Score (XGB-EWS) incorporating vital signs, demographics, laboratory markers, plus medical history embeddings extracted using sentence transformers. We used propensity score weighting to mitigate intervention bias and evaluated performance using net benefit, area under the receiver operating characteristic curve, and calibration. This study included data for 2 080 765 hospital encounters representing 825 200 unique patients over 5 years. The majority (50·9%) of the weighted population was aged 18-65 years, 50·4% were female, 49·6% were male, and 66·2% were admitted to internal and acute medicine clinical specialties. Decision curve analysis in the overall population showed maximum net benefit differences of 1·9 additional correct mortality identifications per 10 000 patients between XGB-EWS and NEWS, and 1·5 per 10 000 patients between NEWS and simplified NEWS. However, stratified analyses revealed substantial heterogeneity: in the high-risk group (initial NEWS ≥7), XGB-EWS provided a maximum gain of 78·1 correct identifications per 10 000 patients compared with NEWS, whereas simplified NEWS resulted in a maximum performance loss of 60·2 per 10 000 patients. Conversely, in the low-risk group (NEWS 0-4), differences were minimal: XGB-EWS provided a maximum gain of 0·9 correct identifications per 10 000 patients compared with NEWS, while simplified NEWS resulted in a maximum loss of 0·1 per 10 000 patients.
How It Works, Simply
This ai in medicine research focuses on AI model. Researchers evaluated it using an eHR Cohort study + Clinical AI evaluation (000 patients) at the real-world cohort stage of medical research.
3 Key Takeaways
Clinical Takeaways- 01Core Finding
In this multicentre retrospective cohort study, we included data from 15 Danish hospitals (ten hospital networks) across the Capital and Zealand regions of Denmark, between Oct 16, 2018, and Oct 16, 2023.
- 02Study Evidence
Tested in a ehr cohort study + clinical ai evaluation (000 patients) at the real-world cohort stage.
- 03Practical Meaning
AI model accuracy depends on training dataset diversity and requires prospective validation across independent hospitals before routine rollout.
Biological Mechanism & Study Design
Study Design & Cohort
EHR Cohort study + Clinical AI evaluation
EHR Cohort study + Clinical AI evaluation (000 patients) · Real-World Cohort
Biological Pathway & Modality
AI model
Intervention: Development and validation of early warning scores predicting 24 h mortality
Experimental Findings & Endpoints
The National Early Warning Score (NEWS) is widely used to predict patient deterioration, but retrospective validations face intervention bias and emphasise discrimination performance over clinical utility.
We aimed to compare the clinical net benefit of NEWS against simplified scoring rules and machine learning models, to determine whether simpler approaches suffice or if complex models offer meaningful advantages, and to evaluate performance heterogeneity across patient subgroups. In this multicentre retrospective cohort study, we included data from 15 Danish hospitals (ten hospital networks) across the Capital and Zealand regions of Denmark, between Oct 16, 2018, and Oct 16, 2023.
We included all adults (aged ≥18 years) with a hospital encounter with at least one NEWS assessment. Demographic and clinical data, NEWS data, and time of death were extracted from hospital and regional electronic health records.
Patients were followed up until death or completion of the 24 h observation period after the initial NEWS score was recorded, whichever occurred first. The outcome was all-cause mortality within 24 h of the initial NEWS recording.
We compared NEWS against both simpler and more complex approaches for predicting 24 h mortality: simplified NEWS (NEWS without blood pressure and temperature), Demographic Early Warning Score (DEWS; simplified NEWS with age and sex), and a model based on Extreme Gradient Boosting Early Warning Score (XGB-EWS) incorporating vital signs, demographics, laboratory markers, plus medical history embeddings extracted using sentence transformers. We used propensity score weighting to mitigate intervention bias and evaluated performance using net benefit, area under the receiver operating characteristic curve, and calibration.
This study included data for 2 080 765 hospital encounters representing 825 200 unique patients over 5 years. The majority (50·9%) of the weighted population was aged 18-65 years, 50·4% were female, 49·6% were male, and 66·2% were admitted to internal and acute medicine clinical specialties.
Decision curve analysis in the overall population showed maximum net benefit differences of 1·9 additional correct mortality identifications per 10 000 patients between XGB-EWS and NEWS, and 1·5 per 10 000 patients between NEWS and simplified NEWS. However, stratified analyses revealed substantial heterogeneity: in the high-risk group (initial NEWS ≥7), XGB-EWS provided a maximum gain of 78·1 correct identifications per 10 000 patients compared with NEWS, whereas simplified NEWS resulted in a maximum performance loss of 60·2 per 10 000 patients.
Conversely, in the low-risk group (NEWS 0-4), differences were minimal: XGB-EWS provided a maximum gain of 0·9 correct identifications per 10 000 patients compared with NEWS, while simplified NEWS resulted in a maximum loss of 0·1 per 10 000 patients.
Clinical Translation & Practice Context
Model complexity produced small average clinical utility gains, concentrated among patients initially classified as high risk by NEWS.
These findings support prospective trials of risk-stratified monitoring based on the first NEWS assessment, testing simplified approaches for lower-risk patients while retaining full NEWS or electronic health record models for higher-risk patients.
The Bottom Line · Clinical Pearl
“Model complexity produced small average clinical utility gains, concentrated among patients initially classified as high risk by NEWS.”
Published in The Lancet Digital Health · Current Stage: Real-World Cohort
