Author(s)/Creator(s)

Mary E. HelanderFollow

ORCID

Mary E. Helander: 0000-0002-5185-6867

Document Type

Article

Date

8-23-2024

Keywords

binary classification, health outcome prediction, NEMSIS, population health patterns, MLB method, machine learning evaluation

Language

English

Disciplines

Emergency Medicine | Other Medicine and Health Sciences | Other Social and Behavioral Sciences | Public Health

Description/Abstract

OBJECTIVE: The broad absence of definitive patient outcomes in the NEMSIS public release data hinders research that seeks to understand the impact of pre-hospital care, operations, and overall patterns of population health – including geospatial and demographic differences. This study evaluated the recently proposed binary end-of-event outcome indicator to provide additional validity of the method, to evangelize its employment for more studies to analyze survival impact following an emergency medical event, and to identify appropriate use and interpretation given imperfection in predicted outcomes. METHODS: A recently published binary end-of-event outcome indicator was applied to datasets for each year from 2017 to 2022. Produced indicators were adjusted to address the method's inconsistencies. An array of established performance metrics from the binary classification in the machine learning literature were applied and interpreted. RESULTS: Over-fitting was detected for year 2018, as well as a degradation in performance when applying the method for datasets from year to year. Extended metrics revealed the method's weakness in accurately indicating the minority class: e.g., after adjustments for conflicting labels, “Dead” prediction accuracy was 77.7% for 2018 and 61.8% over the six-year NEMSIS sub-sample, verses 98.8% overall. CONCLUSIONS: After reproducing and then replicating a previously proposed method for predicting NEMSIS binary end-of-event outcomes, this study shows that it produces reasonably good “Dead” or “Alive” indicators. Reporting True Positive Rate (“Dead” prediction accuracy) and True Negative Rate (“Alive” prediction accuracy) is recommended whenever the method is used in NEMSIS analyses. For certain analyses, outcomes at the individual-level may be more appropriately quantified as probabilities using methods such as logistic regression, instead of predicted binary indicators. In the field, more attention to PCR completion of NEMSIS elements eOutcome.01 and eOutcome.02, whenever possible, can significantly enhance the public research datasets.

Additional Information

This is the Author's Original Manuscript (AOM). The Version of Record (VOR) of this manuscript has been published and is available Prehospital Emergency Care (Taylor & Francis), August 23, 2024, DOI: 10.1080/10903127.2024.2389551

Source

submission

Creative Commons License

Creative Commons Attribution 4.0 International License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Share

COinS