Letter to the Editor regarding the article: "identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study"
Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine volume 25, Article number: 113 (2017)
The aim of this Letter to the Editor was to report some methodological shortcomings in a recently published article. Issues regarding missing values and overfitting are mentioned. First, Complete Case (CC) analysis was used instead of an imputation method. Second, there was a high chance of overfitting and lack of model validation. In conclusion, the results of this study should be interpret with caution and further research is necessary.
With great interest we read the study by Thompson et al.  where they identified pre-hospital factors associated with major trauma outcomes. This study showed that Glasgow Coma Score (GCS), Respiration Rate (RR) and Age are potential predictive triggers for direct transport to a Major Trauma Center (MTC). This is an interesting finding, which might help in the challenging decision which patients will benefit from treatment in MTCs. However, some methodological issues should be taken into consideration.
First, the authors used ‘listwise’ exclusion, also known as complete case (CC) analysis, to handle their missing data. This resulted in excluding almost 45% of their entire sample (462 out of 1033 casualties). Obviously, this leads to less efficiency and possibly bias [2, 3]. In dealing with missing data, the CC analysis could be biased when Missing at Random (MAR) on the outcome variable is present [2,3,4], for example when GCS is missing in patients with high GCS who have a high probability to die. Thus imputation methods should have been considered. This could certainly increase efficiency and potentially reduce bias dependent on the mechanism of missing data [2,3,4].
Second, the authors stated that their model including GCS, RR and Age correctly predicted 97.4% of the casualties. This high prediction rate could be the result of overfitting and might not be generalizable to the population . Three strategies could be followed to avoid overfitting. First, the use of a more liberal p-value than 0.050 and preselection of variables based on clinical knowledge could have decreased the chance of testimation bias and overestimation of the effect of the selected predictors, especially with few events [3, 5]. Two other important steps in prediction modelling are internal and external validation [3, 6]. Internal validation is about the stability of the selected predictors and the quality of the predictions in the underlying population [3, 6]. External validation is about the generalizability of the predictors and predictions in comparable populations [3, 6]. Unfortunately none of these strategies to decrease overfitting and increase the quality of the prediction models have been used and therefore overfitting is likely in this study.
In conclusion, results of this study should be interpret with caution and further research is necessary to estimate the predictive ability of pre-hospital factors with special emphasis on model validity and overfitting.
Glasgow coma score
Missing at random
Major trauma center
Thompson L, Hill M, Davies C, Shaw G, Kiernan MD. Identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study. Scand J Trauma Resusc Emerg Med. 2017;25(1):83.
White IR, Carlin JB. Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Stat Med. 2010;29(28):2920–31.
Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2008.
Vach W, Blettner M. Missing data in epidemiologic studies. Encyclopedia of biostatistics. New Jersey: Wiley; 2005.
Harrell FE. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Cham: Springer International Publishing; 2015.
Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. 2014;35(29):1925–31.
No funding was sought or obtained for this study.
Availability of data and materials
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sewalt, C.A., Wiegers, E.J.A., Venema, E. et al. Letter to the Editor regarding the article: "identifying pre-hospital factors associated with outcome for major trauma patients in a regional trauma network: an exploratory study". Scand J Trauma Resusc Emerg Med 25, 113 (2017). https://doi.org/10.1186/s13049-017-0454-1
- Predictive factors
- Prediction modelling