ORCID
Mary E. Helander: 0000-0002-5185-6867
Document Type
Article
Date
8-23-2024
Keywords
binary classification, health outcome prediction, NEMSIS, population health patterns, MLB method, machine learning evaluation
Language
English
Disciplines
Emergency Medicine | Other Medicine and Health Sciences | Other Social and Behavioral Sciences | Public Health
Description/Abstract
OBJECTIVE: The broad absence of definitive patient outcomes in the NEMSIS public release data hinders research that seeks to understand the impact of pre-hospital care, operations, and overall patterns of population health – including geospatial and demographic differences. This study evaluated the recently proposed binary end-of-event outcome indicator to provide additional validity of the method, to evangelize its employment for more studies to analyze survival impact following an emergency medical event, and to identify appropriate use and interpretation given imperfection in predicted outcomes. METHODS: A recently published binary end-of-event outcome indicator was applied to datasets for each year from 2017 to 2022. Produced indicators were adjusted to address the method's inconsistencies. An array of established performance metrics from the binary classification in the machine learning literature were applied and interpreted. RESULTS: Over-fitting was detected for year 2018, as well as a degradation in performance when applying the method for datasets from year to year. Extended metrics revealed the method's weakness in accurately indicating the minority class: e.g., after adjustments for conflicting labels, “Dead” prediction accuracy was 77.7% for 2018 and 61.8% over the six-year NEMSIS sub-sample, verses 98.8% overall. CONCLUSIONS: After reproducing and then replicating a previously proposed method for predicting NEMSIS binary end-of-event outcomes, this study shows that it produces reasonably good “Dead” or “Alive” indicators. Reporting True Positive Rate (“Dead” prediction accuracy) and True Negative Rate (“Alive” prediction accuracy) is recommended whenever the method is used in NEMSIS analyses. For certain analyses, outcomes at the individual-level may be more appropriately quantified as probabilities using methods such as logistic regression, instead of predicted binary indicators. In the field, more attention to PCR completion of NEMSIS elements eOutcome.01 and eOutcome.02, whenever possible, can significantly enhance the public research datasets.
Recommended Citation
Helander, Mary E. “‘Dead or Alive?’ Assessment of the Binary End-of-Event Outcome Indicator for the NEMSIS Public Research Dataset.” Prehospital Emergency Care, 2024, pp. 1–15, https://doi.org/10.1080/10903127.2024.2389551
Source
submission
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Included in
Emergency Medicine Commons, Other Medicine and Health Sciences Commons, Other Social and Behavioral Sciences Commons, Public Health Commons
Additional Information
This is the Author's Original Manuscript (AOM). The Version of Record (VOR) of this manuscript has been published and is available Prehospital Emergency Care (Taylor & Francis), August 23, 2024, DOI: 10.1080/10903127.2024.2389551