A validation of machine learning-based risk scores in the prehospital setting.

Publisher: Public Library of Science Country of Publication: United States NLM ID: 101285081 Publication Model: eCollection Cited Medium: Internet ISSN: 1932-6203 (Electronic) Linking ISSN: 19326203 NLM ISO Abbreviation: PLoS One Subsets: MEDLINE

Original Publication: San Francisco, CA : Public Library of Science

Machine Learning*; Ambulances/*statistics & numerical data ; Critical Care/*standards ; Emergency Service, Hospital/*statistics & numerical data ; Needs Assessment/*statistics & numerical data ; Risk Assessment/*methods ; Triage/*methods; Aged ; Female ; Hospitalization ; Humans ; Male ; Middle Aged ; Prospective Studies ; ROC Curve ; Sweden

Background: The triage of patients in prehospital care is a difficult task, and improved risk assessment tools are needed both at the dispatch center and on the ambulance to differentiate between low- and high-risk patients. This study validates a machine learning-based approach to generating risk scores based on hospital outcomes using routinely collected prehospital data.
Methods: Dispatch, ambulance, and hospital data were collected in one Swedish region from 2016-2017. Dispatch center and ambulance records were used to develop gradient boosting models predicting hospital admission, critical care (defined as admission to an intensive care unit or in-hospital mortality), and two-day mortality. Composite risk scores were generated based on the models and compared to National Early Warning Scores (NEWS) and actual dispatched priorities in a prospectively gathered dataset from 2018.
Results: A total of 38203 patients were included from 2016-2018. Concordance indexes (or areas under the receiver operating characteristics curve) for dispatched priorities ranged from 0.51-0.66, while those for NEWS ranged from 0.66-0.85. Concordance ranged from 0.70-0.79 for risk scores based only on dispatch data, and 0.79-0.89 for risk scores including ambulance data. Dispatch data-based risk scores consistently outperformed dispatched priorities in predicting hospital outcomes, while models including ambulance data also consistently outperformed NEWS. Model performance in the prospective test dataset was similar to that found using cross-validation, and calibration was comparable to that of NEWS.
Conclusions: Machine learning-based risk scores outperformed a widely-used rule-based triage algorithm and human prioritization decisions in predicting hospital outcomes. Performance was robust in a prospectively gathered dataset, and scores demonstrated adequate calibration. Future research should explore the robustness of these methods when applied to other settings, establish appropriate outcome measures for use in determining the need for prehospital care, and investigate the clinical impact of interventions based on these methods.
Competing Interests: The authors have declared that no competing interests exist.

Resuscitation. 2015 Apr;89:31-5. (PMID: 25583148)
Prehosp Disaster Med. 2002 Jul-Sep;17(3):167-9. (PMID: 12627921)
Crit Care. 2019 Feb 22;23(1):64. (PMID: 30795786)
Acad Emerg Med. 2004 Mar;11(3):312-5. (PMID: 15001417)
Resuscitation. 2019 May;138:322-329. (PMID: 30664917)
J Am Geriatr Soc. 2013 Jan;61(1):12-7. (PMID: 23311549)
Injury. 2011 Sep;42(9):889-95. (PMID: 20430387)
BMJ. 2011 Jun 01;342:d2983. (PMID: 21632665)
Qual Saf Health Care. 2004 Oct;13(5):363-73. (PMID: 15465940)
Resuscitation. 2018 Nov;132:101-111. (PMID: 30171976)
Scand J Trauma Resusc Emerg Med. 2018 Nov 9;26(1):94. (PMID: 30413213)
Ann Emerg Med. 2018 May;71(5):565-574.e2. (PMID: 28888332)
JAMIA Open. 2019 Jul 01;2(3):346-352. (PMID: 31984367)
Prehosp Emerg Care. 2010 Jul-Sep;14(3):329-33. (PMID: 20507220)
BMJ. 2008 Sep 22;337:a1501. (PMID: 18809587)
Med J Aust. 2011 Jun 6;194(11):574-8. (PMID: 21644869)
J Emerg Trauma Shock. 2018 Apr-Jun;11(2):111-114. (PMID: 29937640)
Prehosp Emerg Care. 2013 Jan-Mar;17(1):29-37. (PMID: 23140195)
Int Emerg Nurs. 2016 May;26:32-7. (PMID: 26472522)
Acad Emerg Med. 2006 Mar;13(3):314-24. (PMID: 16495420)
J Am Coll Surg. 2010 Feb;210(2):220-7. (PMID: 20113943)
JAMA. 2010 Aug 18;304(7):747-54. (PMID: 20716737)
Ann Emerg Med. 2010 Oct;56(4):341-7. (PMID: 20554351)
Intern Emerg Med. 2015 Mar;10(2):171-5. (PMID: 25446540)
Emerg Med J. 2004 Jan;21(1):115-8. (PMID: 14734398)
Ann Emerg Med. 2019 Sep;74(3):345-356. (PMID: 31229391)
BMC Emerg Med. 2015 Oct 27;15:32. (PMID: 26507648)
Prehosp Emerg Care. 2006 Apr-Jun;10(2):213-9. (PMID: 16531379)
J Emerg Med. 2016 Jun;50(6):910-8. (PMID: 27133736)
J Am Coll Surg. 2016 Jun;222(6):1125-37. (PMID: 27178369)
Am J Emerg Med. 2018 Sep;36(9):1650-1654. (PMID: 29970272)
Aust Health Rev. 2016 Sep;40(4):378-384. (PMID: 26568037)
Emerg Med J. 2007 Oct;24(10):727-9. (PMID: 17901280)
Eur J Trauma Emerg Surg. 2011 Feb;37(1):73-8. (PMID: 26814754)
Am Heart J. 2013 Mar;165(3):363-70. (PMID: 23453105)
Ann Intern Med. 2015 Jan 6;162(1):55-63. (PMID: 25560714)
Acad Emerg Med. 2006 Aug;13(8):879-85. (PMID: 16825670)
PLoS One. 2018 Jul 20;13(7):e0201016. (PMID: 30028888)
JAMA Intern Med. 2018 Nov 1;178(11):1544-1547. (PMID: 30128552)
Med J Aust. 2011 Feb 21;194(4):S34-7. (PMID: 21401486)
BMJ Open. 2018 Oct 27;8(10):e022528. (PMID: 30368449)
South Med J. 2015 Jul;108(7):381-6. (PMID: 26192931)
Prehosp Emerg Care. 2020 Mar-Apr;24(2):273-281. (PMID: 31210571)
J Surg Res. 2011 Mar;166(1):40-4. (PMID: 20828742)
J Clin Nurs. 2019 Jan;28(1-2):235-244. (PMID: 30016570)
NPJ Digit Med. 2018 May 8;1:18. (PMID: 31304302)
PLoS One. 2017 Nov 22;12(11):e0188532. (PMID: 29166411)
Emerg Med J. 2019 Feb;36(2):97-106. (PMID: 30510034)
Acta Anaesthesiol Scand. 2019 May;63(5):676-683. (PMID: 30623422)
Health Aff (Millwood). 2013 Dec;32(12):2116-21. (PMID: 24301394)
Emerg Med J. 2008 Nov;25(11):773-6. (PMID: 18955625)

Date Created: 20191214 Date Completed: 20200402 Latest Revision: 20231020

20231215

PMC6910679

10.1371/journal.pone.0226518

31834920