Skip to main content
Download PDF
  • Original research
  • Open access
  • Published:

Identification and internal validation of models for predicting survival and ICU admission following a traumatic injury

Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine volume 26, Article number: 95 (2018) Cite this article

Abstract

Background

Measures to improve the accuracy of determining survival and intensive care unit (ICU) admission using the International Classification of Injury Severity Score (ICISS) are not often conducted on a population-wide basis. The aim is to determine if the predictive ability of survival and ICU admission using ICISS can be improved depending on the method used to derive ICISS and incremental inclusion of covariates.

Method

A retrospective analysis of linked injury hospitalisation and mortality data during 1 January 2010 to 30 June 2014 in New South Wales, Australia was conducted. Both multiplicative-injury and single-worst-injury ICISS were calculated. Logistic regression examined 90-day mortality and ICU admission with a range of predictor variables. The models were assessed in terms of their ability to discriminate survivors and non-survivors, model fit, and variation explained.

Results

There were 735,961 index injury admissions, 13,744 (1.9%) deaths within 90-days and 23,054 (3.1%) ICU admissions. The best predictive model for 90-day mortality was single-worst-injury ICISS including age group, gender, all comorbidities, trauma centre type, injury mechanism, and nature of injury as covariates. The multiplicative-injury ICISS with age group, gender, all comorbidities, injury mechanism, and nature of injury was the best predictive model for ICU admission.

Conclusions

The inclusion of comorbid conditions, injury mechanism and nature of injury, improved discrimination for both 90-day mortality and ICU admission. Moves to routinely use ICD-based injury severity measures, such as ICISS, should be considered for hospitalisation data replacing more resource-intensive injury severity classification measures.

Background

The International Classification of Injury Severity Score (ICISS) is one of a number of indices that can be used to estimate injury severity [1,2,3,4]. The ICISS has previously been shown to provide good estimates of survival following injury [1, 5]. It also has a practical advantage [6] as it is relatively easily derived by calculating Survival Risk Ratios (SRRs) for injury diagnosis classifications using hospital administrative data and an indicator of mortality. Both in-hospital [5, 7] and post-discharge mortality up to 90-days post-admission [8] have been used to indicate survival. The ICISS has largely been derived using data from trauma centre registries [2, 3] which generally record information on more severely injured individuals, and have higher mortality rates (around 5%) compared to all injury admissions (around 1–2%) [7, 8]. In addition, the ICISS has been estimated using pooled diagnosis-specific survival probabilities from hospitalisation data across several high-income countries using in-hospital mortality to indicate survival [9].

Assessments of the predictive ability of ICISS values to determine survival have found that the SRR of the worst-injury is often a better predictor of survival than a multiplicative combination of SRRs for all injury diagnoses [10, 11]. Previous studies have also examined the inclusion of other factors that may impact on survival, such as age, gender, pre-injury comorbid conditions, and mechanism of injury. None have considered whether including type of trauma service improves predictive ability [1,2,3, 7, 8, 10]. Previous research has indicated that treatment at a major trauma service can provide a survival advantage [12, 13], but it is unclear whether the accuracy of ICISS to indicate post-discharge survival could be improved by including type of trauma service as a covariate.

Various factors are associated with an injured individual being admitted to an Intensive Care Unit (ICU), including age, gender, comorbid health conditions, and injury mechanism [14,15,16]. However, there has been limited examination of ICISS as a tool to assist in predicting ICU admission [5]. Predicting which patients may require admission to ICU based on their injury severity will assist in determining resource use and is an additional health outcome indicator. Assessment of whether multiplicative-injury or worst-injury ICISS is a better predictor of ICU admission for trauma patients on a population-wide basis is needed.

Different approaches have previously been used to internally validate SRRs estimated from in-hospital records. These have largely involved using the split-sample approach, where the data are randomly spilt into two (or more) parts, with one dataset acting as the training dataset where the model is developed, and the other dataset(s) acting as the testing dataset(s) where predictive accuracy is assessed [6, 10], or using a bootstrapping approach where the model is developed on the full dataset and bootstrapping is applied to assess performance [7, 8]. Limitations of the split-sample approach include creation of imprecise models [17,18,19], underestimated performance of the full model [17], non-efficient use of all data [17, 20], adverse effects on calibration [4, 21], and that validation only occurs on a sample of the full-dataset [20]. As a result of the limitations with the spilt-sample approach for internal validation of prognostic models, bootstrapping has been recommended as the preferred approach for assessing the internal validation of predictive models [17, 18, 20,21,22,23,24]. The bootstrapping approach is able to provide optimism-corrected estimates of the fit statistics. A previous comparison of the bootstrapping approach using a split-sample to predict 30-day mortality following acute myocardial infarction identified that internal validity was best estimated with bootstrapping as it provided the most stable estimates with low bias [17]. This research aims to determine if the predictive ability of survival and ICU admission using ICISS can be improved depending on the method used to derive ICISS and incremental inclusion of covariates.

Method

Linked hospitalisation and mortality data

Hospitalisation data included information on all inpatient admissions for all public and private hospitals in New South Wales (NSW), Australia during the period 1 January 2010 to 30 June 2014. Diagnoses and external cause codes were classified using the International Classification of Diseases, 10th Revision, Australian Modification (ICD-10-AM) [25]. Injury-related admissions were identified using a principal diagnosis of injury (ICD-10-AM: S00-T89). Mortality data from 1 January 2010 to 31 March 2015 from the NSW Registry of Births, Deaths and Marriages was probabilistically linked to the hospitalisation records by the Centre for Health Record Linkage.

The state of NSW, Australia covers an area of 800,628 km2 [26], with a population of 7.7 million [27]. NSW has had a trauma management system since 1991. This system facilitates transfer to and optimal treatment of an injured individual at the most appropriate hospital [28]. A major, level 1 trauma service is able to provide the full spectrum of care for severely injured patients from resuscitation to discharge and a regional trauma service is capable of providing care to individuals with minor to moderate injuries [29]. Regional trauma services often provide initial assessment and stabilisation of a seriously injured patient, before transfer to a major trauma service. Within NSW there are ten major trauma centres (including three paediatric) and ten regional trauma centres [30].

All hospital episodes of care related to the one injury event were linked to form a period of care (i.e. all episodes of care related to the injury until discharge from the health system). Ninety-day mortality was calculated from the date of admission of the index injury hospital admission. Where an individual was treated at more than one hospital for their injury, trauma care was considered to be delivered at the hospital where the majority of patient care was provided as defined by length of stay (LOS).

Comorbidity identification

The Charlson Comorbidity Index [31] was used to identify 17 comorbidities using up to 50 diagnosis classifications in the hospitalisation data and a 12-month look-back period to 1 January 2009. Each comorbid condition was assigned a weight between 1 and 6 based on the risk of mortality and/or resource use and the sum of weights was used to generate a comorbidity score. A higher score is indicative of a higher likelihood of mortality and/or resource use. In addition, specific health conditions that are associated with injury risk and poor recovery [32, 33] including mental health conditions (ICD-10-AM: F20-F50), alcohol misuse and dependence (ICD-10-AM: F10, Y90, Y91, Z50.2, Z71.4, Z72.1), and drug-related dependence (ICD-10-AM: F11-F16, F19, Z50.3, Z71.5, Z72.2) were also identified using diagnosis classifications.

Calculation of the international classification of injury severity score

For all of the index injury hospital admissions, a SRR was calculated for each injury diagnosis. A SRR represents the ratio of the number of individuals with each injury diagnosis who did not die to the total number of individuals with the injury diagnosis. The mean number of diagnoses recorded per injured individual was 1.74 (SD = 1.46; range 1–43). For each injury admission, two ICISS values were calculated: (i) multiplicative-injury ICISS where ICISS is the product of all SRRs for each of the individual’s injuries; and (ii) single worst-injury, where ICISS only includes the worst-injury (i.e. the injury diagnosis with the lowest SRR) as the single worst-injury has been shown to have good discriminatory ability for survival [10].

Data analysis

All analyses were performed using SAS version 9.4 [34]. Logistic regression was used to examine both 90-day mortality and ICU admission as outcomes with varying predictor variables: ICISS (i.e. multiplicative or worst-injury), age group (i.e. 0–16, 17–24, 25–44, 45–64, 65–79, ≥80 years), sex, Charlson Comorbidity Index group (i.e. 0, 1–2, 3–4, ≥5 on the comorbidity index score), mental health conditions (i.e. Y/N), alcohol misuse and dependence (i.e. Y/N), drug related dependence (i.e. Y/N), trauma service level (i.e. major trauma, regional trauma, other hospital), injury mechanism, and nature of injury.

The models were assessed in terms of their ability to discriminate survivors and non-survivors using the ROC curve and the concordance statistic (c-statistic), with better discrimination scores having more area under the ROC curve. A ROC of ≥0.7 and < 0.8 as a general indication was considered to provide acceptable discrimination, a ROC of ≥0.8 and < 0.9 was considered to indicate excellent discrimination, and a ROC ≥0.9 was considered to indicate outstanding discrimination [35]. Model fit was examined using the Akaike information criterion (AIC) that indicates how close a statistical model approaches the true distribution, with lower values indicating a better fit. Goodness-of-fit was examined using the Hosmer-Lemeshow (H-L) statistic that compared predicted mortality with actual mortality, with lower values indicating better calibration. Nagelkerke’s R2 was used as a pseudo R-squared to provide additional information regarding goodness-of-fit of the models by indicating the proportion of outcome variance explained, ranging from 0 to 1, with values closer to one indicating higher variance explained by the model.

The full hospital-mortality data extract was used to calculate the SRRs in order to include the widest possible range of injury diagnosis codes. Therefore, to account for possible bias caused by using a single data extract for both development and testing, non-parametric bootstrapping with 200 replications was used to correct for optimism for calculating the fit statistics and 95% confidence intervals [21]. Bootstrapping involved fitting the full model and then deriving the parameter estimates which were applied to the full dataset to obtain the apparent fit statistics. A bootstrap data sample was then generated to derive fit statistics on the sample using the fitted value from the bootstrapped dataset. The fitted value was applied to the original full dataset to derive another set of fit statistics, which were used to calculate the optimism estimates. This process was replicated 200 times and then the average of the optimism estimates were subtracted from the apparent fit statistics [21].

Results

During the study timeframe, there were 735,961 index injury admissions and 13,744 (1.9%) deaths within 90-days of hospital admission. Over half (55.6%) the injury hospitalisations were of males, with 68.0% aged < 65 years. For over three-quarters (79.7%) of hospitalisations there were no Charlson comorbidities identified. However, for individuals ≥65 years, 106,805 (45.3%) had at least one Charlson comorbidity identified. Just over one-quarter (27.8%) of hospital treatment was provided at a level 1 trauma centre. Falls and land transport crashes accounted for 45.7% of all injury mechanisms and fractures (34.1%) were the most common principal nature of injury (Table 1).

Table 1 Demographic characteristics of individuals with an injury-related hospitalisation, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014
Full size table

90-day mortality

The inclusion of additional predictor variables saw improvement in model assessment criteria for both multiplicative-injury and single worst-injury ICISS for 90-day mortality. As predictor variables were added, concordance improved from 0.886 to 0.917 for multiplicative-injury ICISS and from 0.894 to 0.922 for single worst-injury ICISS. The single worst-injury ICISS was identified as a better predictor of 90-day mortality than the multiplicative-injury ICISS. The best discriminatory model was generated using single-worst-injury ICISS, age group, gender, all comorbidities, trauma centre type, injury mechanism, and nature of injury that explained 33% of variation (i.e. model 8). Generally, inclusion of type of trauma centre did not improve concordance over the inclusion of comorbid conditions (Table 2). The calibration curves for 90-day mortality for all models were similar. Calibration was better for lower mortality and was very good below estimated mortality of 30% (Fig. 1).

Table 2 Model performance for ICISS to predict 90-day mortality, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014
Full size table
Fig. 1
figure 1

Calibration curve for best fit for single worst-injury ICISS to predict 90-day mortality, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014. Model includes: age group, gender, Charlson comorbidities, mental health conditions, alcohol misuse and dependence, drug-related dependence, injury mechanism, nature of injury and trauma centre type

Full size image

ICU admission

There were 23,054 (3.1%) ICU admissions for the injured patients. Of those who were admitted to ICU, 58.8% were male, 46.7% were aged 25–64 years, 37.7% had one or more Charlson comorbidities, 27.0% were injured following a fall, 17.2% following self-harm, and 15.9% after a land transport incident. Fractures (24.1%), poisoning (19.5%), and injury to internal organs (18.3%) were the most common nature of the principal injury.

The inclusion of comorbidities, injury mechanism and nature of injury saw improvement in model assessment criteria for multiplicative-injury and single worst-injury ICISS for ICU admission, with concordance improving from 0.763 to 0.856 for multiplicative-injury ICISS and from 0.745 to 0.841 for single worst-injury ICISS. The multiplicative-injury ICISS was identified as a better predictor of ICU admission compared to the single worst-injury ICISS. The best discriminatory model was generated using multiplicative-injury ICISS, age group, gender, all comorbidities, injury mechanism, and nature of injury which explained 25% of variation (model 7, Table 3). Calibration was better for lower ICU admission and was very good below estimated ICU admission of 40% (Fig. 2).

Table 3 Model performance for ICISS to predict ICU admission, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014
Full size table
Fig. 2
figure 2

Calibration curve for best fit for multiplicative-injury ICISS to predict ICU admission, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014. Model includes: age group, gender, Charlson comorbidities, mental health conditions, alcohol misuse and dependence, drug-related dependence, injury mechanism, and nature of injury

Full size image

Discussion

The ability to provide an indication of injury severity for injury hospital admissions is useful to enable surveillance of injury severity for different injury mechanisms, to clinically evaluate injured patient health outcomes, to provide an indication of injury burden in the population, and to inform clinical resource use [7, 8]. This study demonstrated that the single-worst-injury ICISS was a better predictor of 90-day mortality compared to multiplicative-injury ICISS, with the best model incorporating age group, gender, all comorbidities, trauma centre type, injury mechanism, and nature of injury. While the single-worst-injury ICISS provided the best model of 90-day mortality risk, clinical significance of this model compared to the multiplicative ICISS is not expected. However, this does confirm other research which found that an individual’s worst-injury is most influential in predicting mortality risk [10], rather than a combination of multiple injuries.

Discrimination for 90-day mortality was indicated to range from excellent to outstanding, with concordance ranging from 0.886 to 0.922 for all models. These estimates are higher than previous studies that examined the calculation of ICISS using in-hospital mortality for Australian and New Zealand injury hospitalisations using ICD-10-AM [7] and for the United States (US) using National Trauma Data Bank records and ICD-9-CM [10]. Differences in concordance scores are likely between the current study and previous Australian/New Zealand and US studies as only in-hospital mortality was considered in the previous work compared to 90-day mortality in the current study, with survival post-discharge likely to provide a better indicator of overall survival. Also, differences in the classification structure between ICD-10-AM and ICD-9-CM will account for some differences in concordance scores between the Australian and US research.

The inclusion of information on comorbidities improved concordance from excellent to outstanding for both the single worst-injury and the multiplicative-injury ICISS. Pre-existing health conditions have previously been demonstrated to increase mortality following injury, even for minor-moderate severity injuries [36]. As all hospitals in one Australian state were included in the current study, the impact of including trauma centre type as a covariate in the assessment of predictors of 90-day mortality was able to be examined. However, the inclusion of type of trauma centre did not improve concordance, model fit or variation for 90-day mortality. This seems counterintuitive and could be due to selection bias, with severely injured individuals more likely to be admitted and/or transferred to level 1 trauma centres for treatment and severely injured individuals less likely to survive their injuries irrespective of where they are treated. While an individual who is transferred to a level 1 trauma centre may provide an indicator of injury severity, the severely injured individual would need to be alive when they reached the higher level of care, otherwise there is potential to introduce immortal time bias [37].

Calibration results for 90-day mortality were better at lower levels of mortality (i.e. ≤30%). This is likely to be due to the majority of hospitalisations having low mortality (i.e. ICISS values near 1). The ICISS generated in this study indicating probability of death was overestimated for higher levels of mortality (i.e. ≥60%). Other studies have also found calibration to be better at lower levels of mortality [7, 8].

In the current study, the best model to predict ICU admission was generated using multiplicative-injury ICISS, age group, gender, all comorbidities, injury mechanism, and nature of injury. The multiplicative-injury method to calculate ICISS assumes that each injury independently affects the outcome, which may not necessarily be so when an individual sustains multiple injuries [7]. However, it is possible that the superiority of the multiplicative-injury method to predict ICU admission could be due to individuals requiring ICU admission being more likely to sustain multiple injuries. Gagne and colleagues [38] have also found that the multiplicative-injury ICISS had the best discriminatory ability for ICU admission for individuals hospitalised with traumatic brain injury. Further improvements in the predictive ability of ICU admission are likely to be gained with the addition of physiologic covariates. In particular, the inclusion of covariates was able to improve the model fit for the prediction of ICU admission.

Compared to previous Abbreviated Injury Scale (AIS)-based estimates of injury severity, injury severity scoring using the ICISS is easier and more accessible [10], able to be generated using routinely collected hospital administrative data, but is reliant on accurate classification of the worst-injury. Other indices of severity, such as the Injury Severity Score (ISS), are more resource intensive, and are used by major trauma centres in NSW. The ISS is based on an assessment of all injuries sustained and is generated using AIS scores. It is calculated as the sum of squares of the single highest AIS score in each of the three most severely injured body regions out of nine body regions [39]. Calculation of ISS requires specialist training of data coders and specialist resources. Moving to the routine clinical use of ICISS to estimate injury severity would seem to be a preferential option as comparisons of ICISS and ISS have indicated that ICISS performs as well or better than ISS [1, 5, 40].

There are several limitations of the current study. No information was available on pre-hospital treatment and care and/or use of emergency transport services that may have aided survival. There was also no information available on physiologic responses, such as Glasgow Coma Score, respiratory rate or systolic blood pressure. There were modest numbers for some injury diagnosis classifications and there is a low proportion of ICU admissions and deaths in the current study, so SRRs were based on small counts. The low proportion of deaths would have influenced the SRRs, as a higher proportion of diagnoses classifications would have survived. Likewise, for ICU admissions, with injured individuals being less likely to be admitted to ICU. However, balancing these limitations, these injuries represented all the hospital admissions and 90-day mortality in the state.

Additionally, no validation of the diagnosis classifications in the hospital administrative records was able to be conducted. This study did not use a split-sample approach on hospital-mortality data extract to develop the SRRs, instead the SRRs were developed on the original dataset to maximise the injury diagnosis classifications available to develop the SRRs, and bootstrapping was applied. The authors did investigate and perform a midpoint time-based split-sample approach for modelling and testing, with little difference in R2 or concordance values for ICISS to predict 90-day mortality (Additional file 1: Table S1), or to predict ICU admission (Additional file 2: Table S2). Only health conditions relevant to the hospitalisation are recorded, so it is possible that the number of health conditions experienced are under-enumerated, even with the 1-year look-back period. A longer lookback period may have been able to provide a better indication of long-term comorbid health conditions [41]. Deaths that occurred prior to hospital admission were not considered. External validation of the SRRs generated in the current study is recommended to ascertain if differences in model performance for 90-day mortality and ICU admission are replicated using the same predictor variables in other jurisdictions. Lastly, ICISS is an indicator of threat-to-life, so does not consider threat to on-going disability following injury.

Conclusion

This study has demonstrated that the worst-injury ICISS was a better predictor of 90-day mortality and that multiplicative-injury ICISS was a better predictor of ICU admission. It also demonstrated better calibration and explained variance for both outcomes with inclusion of covariates, particularly comorbid conditions, injury mechanism and nature of injury. Moves to routinely use ICD-based injury severity measures, such as ICISS, should be considered for hospitalisation data.

Abbreviations

AIC:

Akaike information criterion

AIS:

Abbreviated Injury Scale

H-L:

Hosmer-Lemeshow

ICD:

International Classification of Diseases

ICISS:

International Classification of Injury Severity Score

ICU:

Intensive Care Unit

LOS:

Length of Stay

NSW:

New South Wales

ROC:

Receiver Operator Characteristic

SRR:

Survival Risk Ratio

References

  1. Stephenson S, Langley J, Civil I. Comparing measures of injury severity for use with large databases. J Trauma Inj Inf Crit Care. 2002;53:326–32.

    Article  Google Scholar 

  2. Glance L, Osler T, Maukamel D, Meredith W, Wagner J, Dick A. TMPM-ICD9 a trauma mortality prediction model based on ICD-9-CM codes. Ann Surg. 2009;249(6):1032–9.

    Article  Google Scholar 

  3. Willis C, Gabbe B, Jolley D, Harrison J, Cameron P. Predicting trauma patient mortality: ICD [or ICD 10 AM] versus AIS based approaches. ANZ J Surg. 2010;80(11):802.

    Article  Google Scholar 

  4. Sacco W, MacKenzie E, Champion H, Davis E, Buckman R. Comparison of alternative methods for assessing injury severity based on anatomic descriptors. J Trauma Inj Infect Crit Care. 1999;47(3):441–6.

    Article  CAS  Google Scholar 

  5. Gagne M, Moore L, Beaudoin C, Kuimi B, Sirois M. Performance of international classification of diseases-based injury severity measures used to predict in-hospital mortality: a systematic review and meta-analysis. J Trauma Acute Care Surg. 2016;80(3):419–26.

    Article  Google Scholar 

  6. Meredith J, Kilgo P, Osler T. A fresh set of survival risk ratios derived from incidents in the National Trauma Data Bank from which the ICISS may be calculated. J Trauma Acute Care Surg. 2003;55(5):924–32.

    Article  Google Scholar 

  7. Stephenson S, Henley G, Harrison J, Langley J. Diagnosis based injury severity scaling: investigation of a method using Australian and New Zealand hospitalisations. Injury Prev. 2004;10:379–83.

    Article  CAS  Google Scholar 

  8. Davie G, Cryer C, Langley J. Improving the predictive ability of the ICD-based injury severity score. Injury Prev. 2008;14:250–5.

    Article  CAS  Google Scholar 

  9. Gedeborg R, Warner M, Chen L, Gulliver P, Cryer C, Robitaille Y, Bauer R, Ubeda C, Lauritsen J, Harrison J. Internationally comparable diagnosis-specific survival probabilities for calculation of the ICD-10–based injury severity score. J Trauma Acute Care Surg. 2014;76(2):358–65.

    Article  Google Scholar 

  10. Kilgo P, Osler T, Meredith J. The worst injury predict mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring. J Trauma. 2003;55(4):599–607.

    Article  Google Scholar 

  11. Tepas J, Leaphart C, Celso b, Tuten J, Pieper P, Ramenosfky M. Risk stratification simplified: the worst injury predicts mortality for the injured children. J Trauma. 2008;65:1258–63.

    Article  Google Scholar 

  12. Pracht E, Tepas J, Celso B, Langland-Orban B, Flint L. Survival advantage associated with treatment of injury at designated trauma centers. Med Care Res Rev. 2007;64(1):83–97.

    Article  Google Scholar 

  13. Pracht E, Tepas J, Langland-Orban B, Simpson L, Pieper P, Flint L. Do pediatric patients with trauma in Florida have reduced mortality rates when treated in designated trauma centres? J Pediatr Surg. 2008;43:212–21.

    Article  Google Scholar 

  14. Balogh Z, Varga E, Tomka J, Süveges G, Tóth L, Simonka J. The new injury severity score is a better predictor of extended hospitalization and intensive care unit admission than the injury severity score in patients with multiple orthopaedic injuries. J Orthop Trauma. 2003;17(7):508–12.

    Article  Google Scholar 

  15. Lavoie A, Moore L, LeSage N, Liberman M, Sampalis J. The injury severity score or the new injury severity score for predicting intensive care unit admission and hospital length of stay? Injury. 2005;36(4):477–83.

    Article  Google Scholar 

  16. Tamim H, Al Hazzouri A, Mahfoud Z, Atoui M, El-Chemaly S. The injury severity score or the new injury severity score for predicting mortality, intensive care unit admission and length of hospital stay: experience from a university hospital in a developing country. Injury. 2008;39(1):115–20.

    Article  Google Scholar 

  17. Steyerberg, E., Harrell, F., G., B., Eijkemans, M., Vergouwe, Y., and Habbema, J., Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol, 2001. 54: p. 774–781.

    Article  CAS  Google Scholar 

  18. Steyerberg E, Harrell F. Prediction models need appropriate internal, internal–external, and external validation. J Clin Epidemiol. 2016;69:245–7.

    Article  Google Scholar 

  19. Altman D, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453–73.

    Article  CAS  Google Scholar 

  20. Kononen D, Flannagan C, Wang S. Identification and validation of a logistic regression model for predicting serious injuries associated with motor vehicle crashes. Accid Anal Prev. 2011;43(1):112–22.

    Article  Google Scholar 

  21. Harrell F, Lee K, Mark D. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87.

    Article  Google Scholar 

  22. Bleeker S, Moll H, Steyerberg E, Donders A, Derksen-Lubsen G, Grobbee D, Moons K. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56(9):826–32.

    Article  CAS  Google Scholar 

  23. Steyerberg E, Bleeker S, Moll H, Grobbee D, Moons K. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003;56(5):441–7.

    Article  Google Scholar 

  24. Harrell F. Regression modeling strategies with application sto linear models, logistic and original regression, and survival analysis. In: Springer Series in Statistics. Springer: New York; 2001.

    Google Scholar 

  25. National Centre for Classification in Health. ICD-10-AM. Fifth ed. Sydney: National Centre for Classification in Health; 2006.

    Google Scholar 

  26. Australian Government. Area of Australia - States and Territories. 2017 [cited 2017 24/3/2017]; Available from: http://www.ga.gov.au/scientific-topics/national-location-information/dimensions/area-of-australia-states-and-territories.

  27. Australian Bureau of Statistics. Australian demographic Statistics Cat. no. 3101.0. Canberra: ABS; 2016.

    Google Scholar 

  28. NSW Health Department. Selected Specialty And Statewide Services Plans Number Six. In: NSW Trauma Services. North Sydney: NSW Health Department; 2009.

    Google Scholar 

  29. Royal Australasian College of Surgeons. The Australasian trauma verification program manual. Melbourne: Royal Australasian College of Surgeons; 2009.

    Google Scholar 

  30. NSW Institute of Trauma and Injury Management. NSW Trauma Services. 2017 [cited 2017 24/3/2017]; Available from: https://www.aci.health.nsw.gov.au/get-involved/institute-of-trauma-and-injury-management/clinical/trauma_system/nsw_trauma_system/nsw_trauma_services.

  31. Quan H, Li B, Couris C, Fushimi K, Graham P, Hider P, Januel J, Sundararajan V. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173(6):676–82.

    Article  Google Scholar 

  32. Miller T, Lestina D, Smith GS. Injury risk among medically identified alcohol and drug abusers. Alcohol Clin Exp Res. 2001;25(1):54–9.

    Article  CAS  Google Scholar 

  33. Wan J, Morabito D, Khaw L, Knudson M, Dicker R. Mental illness as an independent risk factor for unintentional injury and injury recidivism. J Trauma Acute Care Surg. 2006;61(6):1299–304.

    Article  Google Scholar 

  34. SAS Institute. SAS: statistical software, version 9.4. Cary: SAS Institute; 2014.

    Google Scholar 

  35. Hosmer D, Lemeshow S. Applied logistic regression. Second ed. New York: John Wiley & Sons; 2000.

    Book  Google Scholar 

  36. Hollis S, Lecky F, Yates D, Woodford M. The effect of pre-existing medical conditions and age on mortality after injury. J Trauma Acute Care Surg. 2006;61(5):1255–60.

    Article  Google Scholar 

  37. Hernán M, Sauer B, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol. 2016;79:70–5.

    Article  Google Scholar 

  38. Gagné M, Moore L, Sirois M, Simard M, Beaudoin C, Kuimi B. Performance of international classification of diseases–based injury severity measures used to predict in-hospital mortality and intensive care admission among traumatic brain-injured patients. J Trauma Acute Care Surg. 2017;82(2):374–82.

    Article  Google Scholar 

  39. Baker S, O'Neil B, Haddon W, Long W. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma Injury Infect Crit Care. 1974;14:187–96.

    Article  CAS  Google Scholar 

  40. Meredith J, Evans G, Kilgo P, MacKenzie E, Osler T, McGwin G, Cohn S, Esposito T, Gennarelli T, Hawkins M, Lucas C, Mock C, Rotondo M, Rue L, Champion H. A comparison of the abilities of nine scoring algorithms in predicting mortality. J Trauma Injury Infect Crit Care. 2002;53(4):621–9.

    Article  Google Scholar 

  41. Preen D, Holman CD, Spilsbury K, Semmens J, Brameld K. Length of comorbidity lookback period affected regression model performance of administrative health data. J Clin Epidemiol. 2006;59(9):940–6.

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank the NSW Ministry of Health for providing access to the Admitted Patient Data Collection, the NSW Registry of Births Deaths and Marriages for providing access to mortality data, the Australian Coordinating Registry for providing access to the cause of death unit record file and the CHeReL for conducting the record linkage. RM is supported by a career fellowship from the New South Wales Ministry of Health under the New South Wales Health Early-Mid Career Fellowships Scheme. JB’s research is funded by various NHMRC grants.

Funding

The research was supported by the New South Wales Health Early-Mid Career Fellowships Scheme.

Availability of data and materials

The data that support the findings of this study are available from the NSW Ministry of Health and the NSW Registry of Births, Deaths and Marriages, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of all data custodians supplying the original data and with approval from the ethics committee.

Author information

Authors and Affiliations

  1. Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Road, Sydney, NSW, 2109, Australia

    Rebecca J. Mitchell, Hsuen P. Ting & Jeffrey Braithwaite

  2. Sydney Medical School, The University of Sydney, Sydney, Australia

    Tim Driscoll

Authors
  1. Rebecca J. Mitchell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hsuen P. Ting
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tim Driscoll
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeffrey Braithwaite
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors were all involved in study concept and design. RM acquired the data, HT conducted the analysis and RM wrote the first draft of the manuscript. All authors were all involved in interpretation of data and critical revision of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Rebecca J. Mitchell.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was provided by the NSW Population and Health Services Research Ethics Committee (2015/08/599). A waiver of consent was granted by the ethics committees.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Table S1. Model performance for ICISS to predict 90-day mortality using split-sample approach, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014 (DOCX 16 kb)

Additional file 2:

Table S2. Model performance for ICISS to predict ICU admission using spilt-sample approach, linked hospitalisation and mortality data, NSW, 1 January 2010 to 30 June 2014 (DOCX 16 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mitchell, R.J., Ting, H.P., Driscoll, T. et al. Identification and internal validation of models for predicting survival and ICU admission following a traumatic injury. Scand J Trauma Resusc Emerg Med 26, 95 (2018). https://doi.org/10.1186/s13049-018-0563-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13049-018-0563-5

Keywords

Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine

ISSN: 1757-7241