The counterintuitive effect of multiple injuries in severity scoring: a simple variable improves the predictive ability of NISS
© Di Bartolomeo et al; licensee BioMed Central Ltd. 2011
Received: 28 January 2011
Accepted: 19 April 2011
Published: 19 April 2011
Injury scoring is important to formulate prognoses for trauma patients. Although scores based on empirical estimation allow for better prediction, those based on expert consensus, e.g. the New Injury Severity Score (NISS) are widely used. We describe how the addition of a variable quantifying the number of injuries improves the ability of NISS to predict mortality.
We analyzed 2488 injury cases included into the trauma registry of the Italian region Emilia-Romagna in 2006-2008 and assessed the ability of NISS alone, NISS plus number of injuries, and the maximum Abbreviated Injury Scale (AIS) to predict in-hospital mortality. Hierarchical logistic regression was used. We measured discrimination through the C statistics, and calibration through Hosmer-Lemeshow statistics, Akaike's information criterion (AIC) and calibration curves.
The best discrimination and calibration resulted from the model with NISS plus number of injuries, followed by NISS alone and then by the maximum AIS (C statistics 0.775, 0.755, and 0.729, respectively; AIC 1602, 1635, and 1712, respectively). The predictive ability of all the models improved after inclusion of age, gender, mechanism of injury, and the motor component of Glasgow Coma Scale (C statistics 0.889, 0.898, and 0.901; AIC 1234, 1174, and 1167). The model with NISS plus number of injuries still showed the best performances, this time with borderline statistical significance.
In NISS, the same weight is assigned to the three worst injuries, although the contribution of the second and third to the probability of death is smaller than that of the worst one. An improvement of the predictive ability of NISS can be obtained adjusting for the number of injuries.
The importance of injury scoring is universally recognized and the literature on the subject is immense. Although the scores based on empirical estimation - e.g. Trauma Mortality Prediction Model  - are finally gaining popularity because they show better prediction [1–3], those based on the expert consensus of the Abbreviated Injury Scale (AIS)  lexicon are still widely used. One of the most popular is the New Injury Severity Score (NISS), which is generally recommended as an improvement over the venerable Injury Severity Score (ISS) [5–10].
The role of multiple injuries in outcome prediction and scoring is important because the majority of patients have more than one injury. For example, only 38.3% of patients of the American National Trauma Data Bank (NTDB) sustained a single injury . The way different scores account for the combined effects of multiple injuries varies widely and is a controversial subject [12, 13] that has begun to be elucidated only recently . The last findings seem to suggest that the impact of several injuries on mortality is actually lower than the sum of impacts of the individual injuries. On the contrary, NISS is the simple sum of the three most serious AIS scores squared.
In this study we describe how, in accordance with the current knowledge, the addition of a simple variable that quantifies the number of injuries significantly improves the predictive ability of NISS for mortality.
This study was conducted using data from the trauma registry of the region Emilia-Romagna (RRTG). Emilia-Romagna is an Italian region with about 4 million inhabitants where a trauma system was instituted in 2006. This system is based on three hubs with a defined area of competence that receive patients from scene and other hospitals according to agreed protocols via a pre-hospital Emergency Medical Service. The RRTG started in October 2006 and receives data from the three hubs and seven additional hospitals. The inclusion criteria are traumatic injuries with ISS>15 or admission to intensive care (ICU). ICU admission is decided by clinical judgment and there are no standard criteria. Patients dead on arrival or early in the Emergency Room are recorded in a separate database - not considered by the present study - because they often lack important information like injury severity. The RRTG collects information on demographics, injury, pre-hospital and hospital clinical course and outcome. Injury severity is coded according to the AIS version '98 by one trained coder per hospital. The training was self-managed by regional authorities, with no official certification by Association for the Advancement of Automotive Medicine.
All 3754 cases of the years 2007-2009 were considered for inclusion. All of the cases of 2 hospitals (n = 1180) had to be excluded, because these hospitals recorded only the ISS and not the AIS codes of each lesion. Patients with burns, asphyxia or drowning and those with age <1 year (n = 86) were also excluded. These exclusions are habitual in studies on trauma mortality prediction modeling because severity indexes may perform differently with these injuries  and because the cut-offs of physiologic variables chosen for adults may not apply to infants. The final number of cases was 2488.
A variable (num_inj) expressing the number of AIS-coded injuries sustained by the patients was created, with three categories: one, two, and three or more injuries. The predictive abilities of NISS alone and NISS + num_inj were assessed and compared. The largest severity measure taken from a patient's set of AIS codes (max AIS) was also computed and its discrimination ability quantified and compared. Hierarchical logistic regression models were built, with in-hospital mortality as the outcome variable and single hospitals as the random effect (random intercept). The interaction between NISS and num_inj was also tested with both Wald statistics and the Likelihood Ratio test. The predictive ability was assessed according to discrimination and calibration. Discrimination was measured with the C statistics, also known as area under the ROC curve. The significance of the differences among ROC areas was also assessed . Calibration was assessed with the Hosmer-Lemeshow (HL) goodness of fit test and Akaike's information criterion (AIC), the lower the better. Both the HL C statistics (based on equally sized groups) and HL H statistics (based on fixed cut-points of the predictions) were calculated. The groups were 10 for the C statistics and between eight and ten for the H statistics, depending on the range of predictions (the cut-points for the predicted probability of death were 10%, 20%... etc.). Because it is recognized that a standard and agreed measure of calibration does not exist , we present calibration also by calibration curves. For simplicity, only curves of equally sized groups for simple and complete models with NISS and NISS+num_inj are shown.
Characteristics of the population.
Survivors (n = 2174)
Non survivors (n = 314)
Difference between survivors and non-survivors*
Age (n = 2488), mean ± SD,
44.0 ± 21.4, 41 (1-93)
61.5 ± 23.9, 70 (1-98)
46.3 ± 22.5, 43 (1-98)
p < 0.01
Gender (n = 2488), No of males (%)
p = 0.06
Mechanism of injury, No (%)
p < 0.01
Missing or unknown
NISS (n = 2488), mean ± SD,
30.00 ± 13.6, 27 (1-75)
44.33 ± 18.4, 43 (1-75)
31.81 ± 15.0, 29 (1-75)
p < 0.01
Motor component of GCS, No (%)
p < 0.01
6 - Obeys
5 - Localizes
4 - Withdraws
3 - Decorticate flexion
2 - Extensor response
1 - Nil
Systolic Blood Pressure, No (%)
ICU admission, No (%)
p < 0.01
ICU stay (days), mean (median)
p < 0.01
Hospital stay (days), mean (median)
p < 0.01
ISS>15, No (%)
p < 0.01
ICU stay (days), mean, median
p < 0.01
Hospital stay (days), mean, median
p < 0.01
Number of injuries, No (%)
p = 0.75
3 or more
Mortality, No (%)
A binary variable expressing whether the two worst injuries belong to the same AIS region or not was also tested in combination with NISS and num_inj.
Missing data were treated with casewise exclusion (0, 34, and 115 exclusions respectively in simple, augmented, and complete models). All the analyses were conducted using STATA 10.
Because of the observational design of the study and the anonymity of the final database, neither patient consent nor approval of ethical committee was necessary.
Table 1 shows the characteristics of the 2488 patients by survival status.
C statistics (95% CI)
P of C statistics comparison*
Akaike's information criterion
11.52 p = 0.24
228.68 p < 0.01
14.69 p = 0.14
p = 0.52
NISS + num_inj
p = 0.52
10.32 p = 0.24
11.96 p = 0.28
18.66 p = 0.04
7.47 p = 0.68
17.51 p = 0.06
NISS + num_inj
7.21 p = 0.72
10.27 p = 0.41
10.69 p = 0.38
12.71 p = 0.24
5.50 p = 0.85
15.87 p = 0.10
NISS + num_inj
4.00 p = 0.94
p = 0.52
7.22 p = 0.70
20.79 p = 0.02
6.92 p = 0.73
19.14 p = 0.03
NISS + num_inj
5.76 p = 0.83
13.68 p = 0.18
Regression coefficients of the variable expressing the number of injuries
P of Wald test
2 injuries vs. 1 injury
3 injuries vs. 1 injury
The odds ratio of the variable measuring whether the two worst injuries belong to the same AIS region was 1.19 (95% CI 0.899-1.582, p = 0.22) in the model with NISS and num_inj. Its inclusion virtually did not change the parameters of the other two variables. The interaction with num_inj was not significant either.
Mortality of patients with similar NISS and different number of injuries
3 or more injuries
We found that the predictive ability of models using NISS as injury-severity index is increased by the addition of a variable that summarizes the number of AIS-coded injuries sustained by the patients. The relationship between multiple injuries is fundamental in severity scoring because, as already said, the majority of patients are multiply injured. With hindsight, all the expert consensus and AIS-based scores devised so far oscillated between different ways to account for the following three factors: the contribution of multiple injuries, the weight assigned to injuries other than the worst, and the contribution of the anatomic regions. ISS postulated that the three most severe injuries determine the risk. At the same time it assigned a considerable discounting to additional injuries by disregarding altogether those of the same region. This rule factored in the importance of belonging to the same or different anatomic regions. NISS maintained the principle of considering the three worst injuries, but abolished any form of discounting and consideration for the regions. Essentially, it maximized the role of multiple injuries by giving to the second and third injury the same importance of the first one. The studies showing that max AIS predicts better than NISS and ISS [11, 19, 20] brought the focus again on the minimal importance of injuries other than the worst one: these lesions are disregarded altogether. The Anatomic Profile (AP) , instead, accounts for all three factors in a sophisticated way; the AP includes all the serious injuries in a given body region and weights head and torso injuries more heavily than those in other body regions. It is difficult to condense decades of literature and say which score and underlying method of computing multiple lesions proved best. In general, AP gave most often the best performances, though did not gain popularity for its complexity, while the ranking of the other simpler scores reversed among different studies (e.g. max AIS resulted worse than ISS and NISS in some cases [13, 22], NISS was not always confirmed better than ISS ). The case-mix upon which any method is tested unarguably plays a role, e.g. as injury severity burden worsens, the worst-injury-only scores are penalized .
Another line of research has revealed that patients with identical ISS/NISS scores resulting from different underlying AIS triplets carry quite different mortality risks [23–25]. The researchers focused on the fact that triplets containing the highest AIS score (e.g. 3-0-0 vs. 2-2-1) invariably carried the highest mortality . In the light of our study, it can also depend on the fact that these triplets were also those with the smallest number of injuries for a same ISS/NISS score.
The recent research on new empirical severity scores [1, 13] has shed further light on the role of multiple injuries by carefully quantifying their effect on outcome. In these papers Osler and colleagues showed that it is worth considering up to five injuries, provided that those additional to the worst are carefully weighted. The weight they found for the additional injuries is approximately half that of the worst one. The authors' clinical interpretation of this statistical finding is biologically plausible: although further injuries increase the probability of death, their contribution to the likelihood of death is reduced. They also found that further discounting of predicted mortality results if the 2 worst injuries occur in the same body region (odds ratio for the two worst injuries belonging to the same AIS region: 0.87, p < 0.01).
The behavior of the variable num_inj in our study seems to confirm most of the previous findings. When adjusting for NISS, having two and more than two injuries instead of one lowers the risk of death to approximately one half and one fifth, respectively (table 3). Table 4 is a practical exemplification of the statistical properties of num_inj reported in table 3.
NISS tends to overestimate the risk of death in case of multiple injuries because it does not apply any discounting to the second and third worst lesions. On the other hand, considering only the most severe injury in absolute (max AIS) or by region (ISS) may cause an underestimation of the same risk, as shown by the lowest discrimination of max AIS in our study and by the studies reporting worse performances of max AIS or ISS compared to NISS [5–10, 19, 26]. The adjustment for the number of injuries may address this limitation of NISS in a maybe unorthodox but seemingly efficacious way, preserving the score's inherent simplicity and consideration for multiple injuries. The downside is that a regression model is needed to incorporate this variable, while clinicians use injury scores also as handy, immediate information. However, multivariate models are often used for research and quality assessment and a variable that is both easy to collect and capable of improving the models could be of interest.
An inevitable limitation of num_inj is that it does not discriminate between degrees of injuries. If the second injury is trivial - e.g. a patient with two lesions with AIS severity of 5 and 1 - the increase in prediction given by num_inj is likely to be negligible. A possible refinement, especially for severely injured populations, could be the adoption of a severity threshold (e.g. ≥2) for AIS codes to be included in num_inj.
The fact that in models completed with physiological variables the gain in prediction brought by num_inj is not statistically significant should not be seen as a serious limitation. Physiological variables describe the actual clinical status of the patient that is an important predictor of final outcome. So it is reasonable that they may 'refine' more imperfect anatomic predictors. Yet physiological information is not always available. Moreover, it is possible that in a larger database the models with num_inj could perform significantly better also if completed with physiological variables and we encourage further research.
In our case-mix max AIS performed worse than both NISS and NISS + num_inj. This leaves one question unanswered, i.e. how NISS with num_inj would perform in a population (presumably less-severely injured than ours) where max AIS works better than NISS. Since we tested the interaction of num_inj with NISS and found it non-significant, it would seem that the effect of num_inj be the same across the whole spectrum of injury severity, but, again, more research is needed to answer the question.
Contrary to the findings of Osler et al. we could not show any effect of the two worst injuries belonging to the same AIS region or not. This contradicts a major tenet in the definitions of polytrauma, i.e. that the involvement of different regions increases mortality . Unfortunately we have no explanation for this counterintuitive finding. The term AIS region refers to the nine anatomic regions of the AIS, but is sometimes confused in literature with ISS region, which is their re-arrangement into six, less homogeneous, regions used for ISS computation. We also tested this latter variable, with results similar to those mentioned (data not reported).
This study has some limitations. The number of cases is small and further studies are awaited to confirm or dispute our findings. Moreover, the cases with ISS<16 are not representative of their entire population because RRTG enrolls them only if admitted to ICU. We chose to include in this study all eligible RRTG patients, subtracting to homogeneity, in order to maximize the number of cases. We recognize, however, that this limits the generalizability of our conclusions, calling for further studies in larger populations.
A further limitation is that although we used NISS for selecting patients according to severity (table 2), the RRTG used ISS to define the inclusion threshold for patients not admitted to ICU. Because ISS underestimates severity compared to NISS, the exclusion of some cases with NISS>15 is likely to have occurred. This has probably made this group not representative of the real population with NISS>15, but is unlikely to have influenced our findings.
Another possible criticism is that our outcome was hospital mortality instead of 30-day mortality, recommended by the Utstein template . Unfortunately we were obliged to do so because data on 30-day mortality become available for research with a great delay in our setting. They were available only for the years 2007 and 2008, and suggested that 30-day mortality is probably lower in our setting (9.9 vs. 12.4). However, hospital mortality has been commonly used in most of the cited studies.
Because the predictive performances of models were determined on the same sample of subjects that was used to construct the model, they were probably overestimated. However, such an overestimation would be common to all models and therefore should not affect the relative comparisons between them, the main goal of the study.
In NISS, the same weight is assigned to the three worst injuries, although the contribution of the second and third to the likelihood of death is smaller than that of the worst one. An improvement of the predictive ability of NISS can be obtained adjusting for the number of injuries.
The authors are grateful to the members of working group on major trauma of the region Emilia-Romagna; without their work, this would not have been possible: Barozzi Marco - Azienda Usl di Modena, Chieregato Arturo - Azienda Usl di Cesena, Corsi Amedeo - Azienda Usl di Rimini, Fabbri Andrea - Azienda Usl di Forlì, Ferrari Annamaria - Azienda Ospedaliera Santa Maria Nuova di Reggio Emilia, Ferri Enrico - Azienda Ospedaliero-Universitaria S. Anna di Ferrara, Gambale Giorgio - Azienda Usl di Forlì, Gamberini Alfio - Azienda Usl di Ravenna, Giugni Aimone - Azienda Usl di Bologna, Gordini Giovanni - Azienda Usl di Bologna, Mergoni Mario - Azienda Ospedaliero - Universitaria di Parma, Pizzamiglio Mario - Azienda Usl di Piacenza, Ravaldini Maurizio - Azienda Usl di Cesena, Targa Luigi - Azienda Usl di Cesena, Trabucco Laura - Azienda Ospedaliera Santa Maria Nuova di Reggio Emilia, Volpi Annalisa - Azienda Ospedaliero-Universitaria di Parma.
Some data from this manuscript were presented at the meeting 'Trauma Update' held in Milan, Italy on December 13 2010.
- Glance LG, Osler TM, Mukamel DB, Meredith W, Wagner J, Dick AW: TMPM-ICD9: a trauma mortality prediction model based on ICD-9-CM codes. Ann Surg. 2009, 249: 1032-1039. 10.1097/SLA.0b013e3181a38f28.View ArticlePubMedGoogle Scholar
- Glance LG, Osler TM, Mukamel DB, Meredith W, Dick AW: Expert consensus versus empiric estimation of injury severity: effect on quality measurement in trauma. Arch Surg. 2009, 144: 326-332. 10.1001/archsurg.2009.8.View ArticlePubMedGoogle Scholar
- Moore L, Lavoie A, Le Sage N, Bergeron E, Emond M, Abdous B: Consensus or data-derived anatomic injury severity scoring?. J Trauma. 2008, 64: 420-6. 10.1097/01.ta.0000241201.34082.d4.View ArticlePubMedGoogle Scholar
- Association for the Advancement of Automotive Medicine, Committee on Injury Scaling: The Abbreviated Injury Scale-- 1998 Revision (AIS-98). 1998, Des Plaines, IL: Association for the Advancement of Automotive MedicineGoogle Scholar
- Osler T, Baker SP, Long W: A modification of the injury severity score that both improves accuracy and simplifies scoring. J Trauma. 1997, 43: 922-925. 10.1097/00005373-199712000-00009.View ArticlePubMedGoogle Scholar
- Brenneman FD, Boulanger BR, McLellan BA, Redelmeier DA: Measuring injury severity: time for a change?. J Trauma. 1998, 44: 580-582. 10.1097/00005373-199804000-00003.View ArticlePubMedGoogle Scholar
- Lavoie A, Moore L, LeSage N, Liberman M, Sampalis JS: The New Injury Severity Score: a more accurate predictor of in-hospi-tal mortality than the Injury Severity Score. J Trauma. 2004, 56: 1312-1320. 10.1097/01.TA.0000075342.36072.EF.View ArticlePubMedGoogle Scholar
- Sacco WJ, MacKenzie EJ, Champion HR, Davis EG, Buckman RF: Comparison of alternative methods for assessing injury severity based on anatomic descriptors. J Trauma. 1999, 47: 441-446. 10.1097/00005373-199909000-00001.View ArticlePubMedGoogle Scholar
- Sullivan T, Haider A, DiRusso SM, Nealon P, Shaukat A, Slim M: Pre- diction of mortality in pediatric trauma patients: new injury severity score outperforms injury severity score in the severely injured. J Trauma. 2003, 55: 1083-1087. 10.1097/01.TA.0000102175.58306.2A.View ArticlePubMedGoogle Scholar
- Tay SY, Sloan EP, Zun L, Zaret P: Comparison of the New Injury Severity Score and the Injury Severity Score. J Trauma. 2004, 56: 162-164. 10.1097/01.TA.0000058311.67607.07.View ArticlePubMedGoogle Scholar
- Gennarelli TA, Wodzin E: AIS 2005: A contemporary injury scale. Injury. 2006, 37: 1083--1091. 10.1016/j.injury.2006.07.009.View ArticlePubMedGoogle Scholar
- Kilgo PD, Osler T, Meredith W: The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring. J Trauma. 2004, 56: 928-934. 10.1097/00005373-200404000-00036.View ArticleGoogle Scholar
- Frankema SP, Steyerberg EW, Harrell FE: The use of single or multiple injury descriptions in the assessment of injury severity. J Trauma. 2004, 56: 928-10.1097/00005373-200404000-00035. author reply 929View ArticlePubMedGoogle Scholar
- Osler T, Glance L, Buzas JS, Mukamel D, Wagner J, Dick A: A trauma mortality prediction model based on the anatomic injury scale. Ann Surg. 2008, 247: 1041-8. 10.1097/SLA.0b013e31816ffb3f.View ArticlePubMedGoogle Scholar
- Krob MJ, D'Amico FJ, Ross DL: Do trauma scores accurately predict outcomes for patients with burns?. J Burn Care Rehabil. 1991, 12: 560-3. 10.1097/00004630-199111000-00011.View ArticlePubMedGoogle Scholar
- DeLong DM, Clarke-Pearson DL: Comparing the areas under two or more correlated receiver operating curves: A nonparametric approach. Biometrics. 1988, 44: 837-845. 10.2307/2531595.View ArticlePubMedGoogle Scholar
- Healey C, Osler TM, Rogers FB, Healey MA, Glance LG, Kilgo PD, Shackford SR, Meredith JW: Improving the Glasgow Coma Scale score: motor score alone is a better predictor. J Trauma. 2003, 54: 671-8. 10.1097/01.TA.0000058130.30490.5D.View ArticlePubMedGoogle Scholar
- Ringdal KG, Coats TJ, Lefering R, Di Bartolomeo S, Steen PA, Røise O, Handolin L, Lossius HM, Utstein TCD expert panel: The Utstein template for uniform reporting of data following major trauma: a joint revision by SCANTEM, TARN, DGU-TR and RITG. Scand J Trauma Resusc Emerg Med. 2008, 16: 7-10.1186/1757-7241-16-7.PubMed CentralView ArticlePubMedGoogle Scholar
- Meredith JW, Evans G, Kilgo PD, MacKenzie E, Osler T, McGwin G, Cohn S, Esposito T, Gennarelli T, Hawkins M, Lucas C, Mock C, Rotondo M, Rue L, Champion HR: A comparison of the abilities of nine scoring algorithms in predicting mortality. J Trauma. 2002, 53: 621-8. 10.1097/00005373-200210000-00001.View ArticlePubMedGoogle Scholar
- Tepas JJ, Leaphart CL, Celso BG, Tuten JD, Pieper P, Ramenofsky ML: Risk stratification simplified: the worst injury predicts mortality for the injured children. J Trauma. 2008, 65: 1258-61. 10.1097/TA.0b013e31818cac29.View ArticlePubMedGoogle Scholar
- Copes WS, Champion HR, Sacco WJ, Lawnick MM, Gann DS, Gennarelli T, MacKenzie E, Schwaitzberg S: Progress in characterizing anatomic injury. J Trauma. 1990, 30: 1200-7. 10.1097/00005373-199010000-00003.View ArticlePubMedGoogle Scholar
- Harwood PJ, Giannoudis PV, Probst C, Van Griensven M, Krettek C, Pape HC, Polytrauma Study Group of the German Trauma Society: Which AIS based scoring system is the best predictor of outcome in orthopaedic blunt trauma patients?. J Trauma. 2006, 60: 334-40. 10.1097/01.ta.0000197148.86271.13.View ArticlePubMedGoogle Scholar
- Russell R, Halcomb E, Caldwell E, Sugrue M: Differences in mortality predictions between Injury Severity Score triplets: a significant flaw. J Trauma. 2004, 56: 1321-4. 10.1097/01.TA.0000062763.21379.D9.View ArticlePubMedGoogle Scholar
- Kilgo PD, Meredith JW, Hensberry R, Osler TM: A note on the disjointed nature of the injury severity score. J Trauma. 2004, 57: 479-85. 10.1097/01.TA.0000141024.96440.7C.View ArticlePubMedGoogle Scholar
- Aharonson-Daniel L, Giveon A, Stein M, Israel Trauma Group (ITG), Peleg K: Different AIS triplets: Different mortality predictions in identical ISS and NISS. J Trauma. 2006, 61: 711-7. 10.1097/01.ta.0000235294.32326.e6.View ArticlePubMedGoogle Scholar
- Hannan EL, Waller CH, Farrell LS, Cayten CG: A comparison among the abilities of various injury severity measures to predict mortality with and without accompanying physiologic information. J Trauma. 2005, 58: 244-251. 10.1097/01.TA.0000141995.44721.44.View ArticlePubMedGoogle Scholar
- Butcher N, Balogh ZY: 51: The definition of polytrauma: the need for international consensus. Injury. 2009, 40 (S4): S12-S22. 10.1016/j.injury.2009.10.032.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.