Development of the major trauma case review tool
Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine volume 25, Article number: 20 (2017)
As many as half of all patients with major traumatic injuries do not receive the recommended care, with variance in preventable mortality reported across the globe. This variance highlights the need for a comprehensive process for monitoring and reviewing patient care, central to which is a consistent peer-review process that includes trauma system safety and human factors. There is no published, evidence-informed standardised tool that considers these factors for use in adult or paediatric trauma case peer-review. The aim of this research was to develop and validate a trauma case review tool to facilitate clinical review of paediatric trauma patient care in extracting information to facilitate monitoring, inform change and enable loop closure.
Development of the trauma case review tool was multi-faceted, beginning with a review of the trauma audit tool literature. Data were extracted from the literature to inform iterative tool development using a consensus approach. Inter-rater agreement was assessed for both the pilot and finalised versions of the tool.
The final trauma case review tool contained ten sections, including patient factors (such as pre-existing conditions), presenting problem, a timeline of events, factors contributing to the care delivery problem (including equipment, work environment, staff action, organizational factors), positive aspects of care and the outcome of panel discussion. After refinement, the inter-rater reliability of the human factors and outcome components of the tool improved with an average 86% agreement between raters.
This research developed an evidence-informed tool for use in paediatric trauma case review that considers both system safety and human factors to facilitate clinical review of trauma patient care.
This tool can be used to identify opportunities for improvement in trauma care and guide quality assurance activities. Validation is required in the adult population.
As many as half of all patients with major traumatic injuries do not receive ideal care, with between 2.5% and 14% of medical errors in trauma deaths determined as potentially clinically preventable . The 2014 Australian Trauma Registry report demonstrated a variance in mortality rates between states and hospitals , such variance is reported across clinical specialties and the globe [3, 4]. The literature also suggests that there is great variability in the quality of care for injured youth [5, 6], that deficiencies exist in the quality of care for 8% to 45% of severely injured children, and that 6% to 32% of in-hospital deaths are preventable . This variance in in-hospital mortality between hospitals highlights the need for a systematic, comprehensive system for monitoring and reviewing patient care to inform processes for change to ultimately improve patient outcomes.
Trauma centres have a wide and varying range of trauma quality improvement projects, and initiatives, including morbidity and mortality meetings. Such meetings are longstanding throughout healthcare for review of patient deaths and complications, however, there remains a need to standardise the approach taken to review cases across hospitals and other trauma care providers . This approach ought not to be hospital-centric and silo driven and should include elements of trauma system safety and human factors . A process with standardized criteria (definitions) and measures of trauma care quality, along with consistent approaches to measurement, monitoring and reporting between hospitals is required to ascertain areas for improvements in care and identify corrective strategies. Further, the impacts of the trauma system should include all phases of care including prehospital, trauma triage criteria, hospital type and interfacility transfer, focusing on timeliness and appropriateness of care .
Within hospitals, there have been a range of classification frameworks and taxonomic tools developed to attempt to identify the causal factors of adverse events from a human factors perspective . Many of the frameworks developed have been based on Reason’s Organisational Accident Causation Model , such as the London Protocol . Several of these do not consider the underlying causes of adverse incidents, however, nor do they consider all human factors, including human behavior (ie. human error), that may contribute towards an adverse event or a potential adverse event occurring.
Currently, there is no published, evidence-informed standardised tool that considers both system safety and human factors for use in adult or paediatric trauma case peer review. To address this evidence-practice gap, the aim of this research was to develop and validate a trauma case review tool to facilitate clinical review of trauma patient care that addresses the recommendations of the World Health Organization (WHO)  and Australasian Trauma Quality Improvement Program Guidelines .
To describe the development and validation of a trauma case review tool to facilitate peer-review of adverse events in paediatric major trauma designed to extract information to facilitate monitoring, inform change and facilitate loop closure.
This trauma case review tool is intended for use to facilitate peer-review of major paediatric trauma cases flagged for analysis as a result of an adverse event and was developed during a state-wide, prospective paediatric trauma system evaluation in Australia’s most populous State, New South Wales . Development was multi-faceted, beginning with a review of the literature on trauma audit tools. Data were extracted from the literature to inform iterative tool development using a consensus approach, which was then followed by pilot and inter-rater reliability testing. Each step is described below.
Review of literature
A review of key principles from the WHO Trauma Quality Improvement Program Guidelines , National Safety and Quality Framework , the Institute of Medicine  and the London Trauma Protocol  along with the international literature on Trauma audit tools was conducted. Electronic database search was conducted using the terms “injury”, “audit”, “tool”, “peer review”.
Development of the trauma case review tool
Extraction of data from the literature identifying categories of factors found to be causally related to adverse events was conducted and a draft tool containing seven components that considered the trauma system, and human factors was developed. A pre-existing, validated, hierarchical human factors framework was included in the tool , consisting of three levels to categorise the human factors contributing to any care delivery problems (Section 6).
The draft tool was reviewed by the NSW Institute of Trauma and Injury Management’s Clinical review committee and then trialed by five experienced trauma clinicians (including a trauma nurse, emergency physicians, and surgeons) using medical records from three de-identified paediatric trauma cases from different hospitals. Following feedback from the reviewers, refinements and retesting of the tool with additional de-identified cases was conducted.
Classification of terms
Information on the role of error in any adverse events was identified for the staff action-related classifications involving medical task failures, monitoring tasks, delays, misdiagnoses, or medication issues. Error was classified using Rasmussen’s  skill, rule or knowledge-based error classifications, or a violation classification . Skill-based errors referred to unintentional failures in the execution of a well-rehearsed action or routine task that required little conscious attention. Rule-based errors referred to unintentional failures during activities conducted in familiar situations that were controlled by stored rules. Knowledge-based errors referred to unintentional failures during a novel situation that required conscious analytic processing and stored knowledge. A violation was considered to be an intentional failure to follow accepted work practices, guidelines or procedures during the execution of a task. It is noted that within this classification system a violation does not indicate the intent to cause harm.
Each of the clinical reviewers was provided verbal instruction on how to use the tool and a data dictionary and each signed a confidentiality agreement. Clarification on definitions and aspects of the tool was provided as required, and modifications made accordingly following the testing.
For the pilot, the reviewers were emailed links to the de-identified files using the secure Cloudstor platform (Australia Academic and Research Network). A second round of testing following refinement of the tool was conducted by eight trauma clinicians who used the tool to classify eight de-identified paediatric trauma cases. This group included the same clinicians who reviewed the pilot tool, as well as interstate and international clinicians from nursing, surgical, emergency and retrieval backgrounds. For the second round of testing, a day long face-to-face meeting was held, orientation to the tool conducted, and hard copies of de-identified trauma cases provided for peer-review. Each reviewer completed the case review independently using the tool. Cases used for testing had been identified as having adverse events by the site trauma service and the age range of the injured children was 8 weeks to 15 years.
Data management and analysis
Data from both rounds of testing were entered in ExcelTM. Double data entry was conducted to ensure accuracy. Inter-rater reliability of the adverse event causal factors and outcomes components sections of the tool was assessed using percent agreement [18, 21]. Entries had to agree exactly to be considered the same, and only those answered as ‘yes/no/not applicable’ were assessed. The percentage agreement was calculated as the ratio of the total number of ‘same’ responses for each data element divided by the total number of data elements assessed. For measurement of concordance in the human factors framework section that consists of three levels to categorise the human factors contributing to any care delivery problems, agreement pertained to raters recording the ‘same’ response for up to three levels for each factor, for example, a Level 1 factor was “Work Environment - Did the work environment contribute to difficulties in delivering the required care?” Level 2 of ‘Work Environment’ is the subsection within Level 1, with a possible response being ‘Light’ and a Level 3 response could have been ‘No or too little light’.
Review of literature
Several papers reported that they conducted a peer-review process and classified outcomes as either preventable or non-preventable [22–24]. No original research studies of a validated peer-review tool for use in trauma care were identified.
The usability testing and pilot of the trauma case review tool by the clinical reviewers identified that several modifications of the tool needed to be made. Modifications made are presented in Table 1.
For the pilot, the interrater reliability of the human factors and outcome components of the tool had an average 81% agreement between raters at level 1, 69% at level 2 and 69% at level 3. After refinement of the tool, the interrater reliability of the human factors and outcome components of the tool improved and had an average 86% agreement between raters at Level 1. There was a moderate decrease in reliability with 67% at level 2 and 63% at level 3.
Final trauma case review tool
The final tool contained ten sections, including patient factors (such as pre-existing conditions), presenting problem, a timeline of events, specific services involved in the care delivery, factors contributing to the care delivery problem (including equipment, work environment, staff action, organizational factors), patient outcome, positive aspects of care and the outcome of panel discussion (Additional files 1 and 2). Each component of the tool was informed by the aforementioned literature search and is outlined in Table 2.
Trauma system components
The first sections (Sections 1–3) of the case review tool include demographic and injury information to allow for the development of a chronology and consideration of age- and any patient- specific physiology . The following sections (Sections 4–5) collect clinical management and service delivery information in the standardised Airway, Breathing, Circulation format known to improve trauma patient outcomes when followed . This clinical categorisation of treatment in common trauma language facilitates case-specific areas of improvement and longer term monitoring of areas of care that may require widespread education or intervention for change [1, 23, 27, 28]. It also enables identification of compliance with specific local or statewide trauma management guidelines or protocols. Detailed information on the locations and services involved in a particular incident or near miss will also allow specific intervention if required, and ongoing monitoring to provide evidence for change.
Consideration of causal factors
Section six records information on human factors that may have influenced clinical practice including: equipment, work environment, staff action, patient, organizational, individual and other factors. The human factors component was adapted from the Human Factors Classification Framework for patient safety . This framework was adopted as previous inter-rater reliability for human factors classifications has been demonstrated to be high  and the approach was based on James Reason’s model of organisational incidents .
The human factors component was used to identify influencing or causal factors that were thought to play a role in leading to the adverse event. Each causal factor was classified into one of seven categories (Fig. 1) then a number of subcategories.
This manuscript describes the development, refinement and reliability of a trauma case review tool to facilitate peer-review of adverse events in pre- and in-hospital care provision for major trauma patients. The major trauma case review tool, informed by evidence, is designed to extract trauma system safety and human factor causal information to facilitate monitoring, inform change and facilitate loop closure in the provision of care of major trauma patients.
This tool incorporates human factors with the intent of enhancing current Morbidity and Mortality review practices. Following an adverse event it is common across industries for the focus to be understanding what happened. The focus in a human factors approach is to understand why an adverse event occurred. Identifying the underlying systemic factors that contributed to the occurrence of an adverse event and the event outcomes will assist in understanding why particular decisions and actions occurred. It can also provide insight as to what can be done to prevent these events from occurring in the future, by addressing the underlying causal factors. Incorporating a human factors review component within this case review tool seeks to place greater focus on why the event occurred. This involves investigating human, equipment, organisational and environmental factors that may have contributed to the occurrence of the event which is in line with WHO recommendations that the review process should identify clinical errors, consider work system factors that contributed to the occurrence of the errors, and facilitate corrective action plans .
Adverse events are directly linked to actual harm resulting from (or omission of) health care and are generally independent of the patient’s disease. An adverse event may occur despite the correct care being given under correct circumstances but ultimately associated with a poor outcome . Errors are one aspect of failure in the processes of care. While historically there has been a focus on the individuals that perpetrated the error we know that human performance and the occurrence of errors is influenced by a range of factors in the environment and organisational systems. It is through understanding these influences that we can identify potential improvements in the system and thereby reduce the re-occurrence of similar incidents in the future. The approach of applying a structured framework of these influencing/causal factors is particularly suited to the case review methodology where the detailed data around individual action may not be available but environmental and organisational factors can be identified by experienced reviewers. This tool records adverse events, but also informs understanding of the areas of the system that require further investigation, and monitoring for trends to highlight where change is needed.
Adverse patient events require monitoring, and, for system-wide change, active monitoring and investigation of local, state-based and a national registry(s) is optimal as they are designed to provide information that can be used to improve the efficiency and quality of trauma care. However, trauma registries require more rigour to be reliable in the quality of the reproducible data [31, 32], and they lack the detail afforded by a human factors enhanced peer-review process. There are growing efforts to improve patient safety in trauma and quantification of the burden of iatrogenic harm could catalyse awareness and stimulate changes in trauma practice and healthcare policy . Future work should include integration of the two.
There are some limitations to the development of the tool. Evaluation of the trauma case review tool in this study was restricted to the paediatric trauma population whose anatomical, physiological and psychological management varies significantly compared to adults [34, 35]. There was a wide range of ages of children examined using the development of the tool, and recognition of these differences in a review tool, including age-specific injury patterns and appropriate care of children’s families should be included [36, 37]. Future clinical care review tools should also include the patient experience where possible, although challenges remain in how best to obtain this information. Doyle et al.  found a positive association between patient experience and measures of the technical quality of care and adverse events and support the inclusion of patient experience as one of the central pillars of quality in healthcare.
This tool was tested using a retrospective review of medical records. This method provided informative timelines and information about procedures and immediate patient outcomes. However it was more difficult to extract human factors information. The tool is recommended to be used in conjunction with staff who were part of the treating team, so they can be involved in the review process to ensure accurate and informed classification of human factors. It is anticipated that more intimate knowledge of situation and organizational factors will allow even more useful information to be captured in the human factors section of the tool. To attempt to link the adverse event to longer term patient outcomes would require linkage with a trauma registry that collects such information or a follow-up study with adversely affected patients.
The next stages of evaluation of the trauma case review tool could include a trial of the tool with an adult population and heuristic evaluation (that is, usability evaluation by a human factors expert against a set of usability rules/principles) and could entail observing clinicians using the tool. This would include consideration of the clinicians’ ability to understand and apply the human factors component of the tool, which requires a degree of understanding of human factors principles. Also, although the case review tool is evidence-informed, and has been piloted by trauma clinicians it requires validation for sensitivity and specificity in identifying causes of adverse events. A retrospective cohort study to measure operating characteristics and prospective implementation of the tool into quality assurance activities to gauge how it is received, how well it identifies adverse events, and what type of quality improvement activities it spurs, would be valuable. Such evaluation could also lead to the development of trauma specific trigger tools, to be used in real time as a predictor for adverse events and provide the basis of a measurable trauma quality improvement program [33, 39].
As many as half of all patients with major traumatic injuries do not receive the recommended care and up to 14% of medical errors in trauma deaths are potentially preventable. This research has developed an evidence-informed tool for use in trauma case review that considers system safety and human factors to facilitate clinical review of trauma patient care. This tool can be used to identify opportunities for improvement in trauma care and guide quality assurance activities.
World Health Organization.
Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. Quality indicators for evaluating trauma care: a scoping review. Arch Surg. 2010;145(3):286–95.
Alfred Health. Caring for the Severely Injured in Australia: Inaugural Report of the Australian Trauma Registry 2010 to 2012. Melbourne: Alfred Health; 2014.
Moore L, Evans D, Hameed SM, et al. Mortality in Canadian Trauma Systems: A Multicenter Cohort Study. Ann Surg. 2017;265(1):212–7.
Makary MA, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016;353:i2139.
Cooper CG, Santana MJ, Stelfox HT. A comparison of quality improvement practices at adult and pediatric trauma centers. Pediatr Crit Care Med. 2013;14(8):e365–71.
Mitchell RJ, Curtis K, Chong S, et al. Comparative analysis of trends in paediatric trauma outcomes in New South Wales, Australia. Injury. 2013;44:97–103.
Stelfox HT, Bobranska-Artiuch B, Nathens A, Straus SE. A systematic review of quality indicators for evaluating pediatric trauma care. Crit Care Med. 2010;38(4):1187–96.
World Health Organization. Guidelines for trauma quality improvement programmes. Geneva: World Health Organization; 2009.
Amalberti R, Benhamou D, Auroy Y, Degos L. Adverse events in medicine: easy to count, complicated to understand, and complex to prevent. J Biomed Inform. 2011;44(3):390–4.
McCarthy A, Curtis K, Holland AJA. Paediatric trauma systems and their impact on the health outcomes of severely injured children: An integrative review. Injury. 2016;47(3):574–85.
Mitchell R, Williamson A, Molesworth B, Chung AZ. A review of the use of human factors classification frameworks that identify causal factors for adverse events in the hospital setting. Ergonomics. 2014;57(10):1443–72.
Reason J. Managing the risks of organisational accidents. Aldershot: Ashgate Publishing Ltd; 1997.
Vincent C, Taylor-Adams S, Chapman E, et al. How to investigate and analyse clinical incidents: Clinical Risk Unit and Association of Litigation and Risk Management Protocol. Br Manage J. 2000;320:777–81.
Curtis K, McCarthy A, Mitchell R, et al. Paediatric trauma systems and their impact on the health outcomes of severely injured children: protocol for a mixed methods cohort study. Scand J Trauma Resusc Emerg Med. 2016;24(1):69.
Australian Commission on Safety and Quality in Health Care. Developing a safety and quality framework for Australia. 2009.
Committee on Quality of Health Care in America Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington: National Academies Press; 2001.
Cole E, Lecky F, West A, et al. The impact of a pan-regional inclusive trauma system on quality of care. Ann Surg. 2016;264(1):188–94.
Mitchell RJ, Williamson A, Molesworth B. Application of a human factors classification framework for patient safety to identify precursor and contributing factors to adverse clinical incidents in hospital. Appl Ergon. 2016;52:185–95.
Rasmussen J. Human errors. A taxonomy for describing human malfunction in industrial installations. J Occup Accid. 1982;4(2):311–33.
Reason J. Human error: models and management. Br Med J. 2000;320(7237):768–70.
Shiloach M, Frencher Jr SK, Steeger JE, et al. Toward Robust Information: Data Quality and Inter-Rater Reliability in the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg. 2010;210(1):6–16.
Shackford SR, Hollingsworth-Fridlund P, McArdle M, Eastman AB. Assuring quality in a trauma system—the medical audit committee: Composition, cost, and results. J Trauma Inj Infect Crit Care. 1987;27(8):866–75.
Gruen RL, Jurkovich GJ, McIntyre LK, Foy HM, Maier RV. Patterns of errors contributing to trauma mortality: Lessons learned from 2594 deaths. Ann Surg. 2006;244(3):371–80.
Jat AA, Khan MR, Zafar H, et al. Peer Review Audit of Trauma Deaths in a Developing Country. Asian J Surg. 2004;27(1):58–64.
Caldwell E, Delprado A, Curtis K. An Overview of Trauma. In: Curtis K, Ramsden C, editors. Emergency and Trauma Care. 2nd ed. Sydney: Elsevier; 2015.
Williams MJ, Lockey AS, Culshaw MC. Improved trauma management with advanced trauma life support (ATLS) training. J Accid Emerg Med. 1997;14:81–3.
Curtis K, Chong S, Mitchell R, Newcombe M, Black D, Langcake M. Outcomes of Severely Injured Adult Trauma Patients in an Australian Health Service: Does Trauma Center Level Make a Difference? World J Surg. 2011;35(10):2332–40.
Palmer CS, Davey TM, Mok MT, et al. Standardising trauma monitoring: The development of a minimum dataset for trauma registries in Australia and New Zealand. Injury. 2012;44(6):834–41.
Mitchell R, Williamson A, Molesworth B. Identifying causal patterns and errors in adverse clinical incidents. Paper presented at: Contemporary Ergonomics and Human Factors 2013: Proceedings of the international conference on Ergonomics & Human Factors 2013, Taylor & Francis: Cambridge; 2013.
Resar R, Rozich J, Classen J. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care 2003. 2003;12(Suppl II):7.
Porgo TV, Moore L, Tardif PA. Evidence of data quality in trauma registries: A systematic review. J Trauma Acute Care Surg. 2016;80(4):648–58.
Moore L, Clark DE. The value of trauma registries. Injury. 2008;39(6):686–95.
Matlow AG, Cronin CMG, Flintoft V, et al. Description of the development and validation of the Canadian Paediatric Trigger Tool. BMJ Qual Safety. 2011;20(5):416–23.
Wyen H, Jakob H, Wutzler S, et al. Prehospital and Early Clinical Care of Infants, Children, and Teenagers Compared to an Adult Cohort: Analysis of 2,961 Children in Comparison to 21,435 Adult Patients from the Trauma Registry of DGU in a 15-Year Period. Eur J Trauma Emerg Surg. 2010;36:300–7.
Zwingmann J, Schmal H, Südkamp NP, Strohm PC. Injury severity and localisations seen in polytraumatised children compared to adults and the relevance for emergency room management. Zentralbl Chir. 2008;133:68.
Andruszkow H, Deniz E, Urner J, et al. Physical and psychological long-term outcome after traumatic brain injury in children and adult patients. Health Qual Life Outcomes. 2014;12:26.
Bulger EM, Kaufman R, Mock C. Childhood crash injury patterns associated with restraint misuse: implications for field triage. Prehosp Disaster Med. 2008;23:9–15.
Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1). doi: 10.1136/bmjopen-2012-001570
Landrigan CP, Stockwell D, Toomey SL, et al. Performance of the global Assessment of Pediatric Patient Safety (GAPPS) Tool. Pediatrics. 2016;137(6):e20154076
The development of this tool was funded by an NHMRC Partnership Grant GNT1092499. Partners include NSW Ambulance, NSW Institute of Trauma and Injury Management, NSW Agency for Clinical Innovation, Thyne Reid Foundation, The Day of Difference Foundation and The NSW Office of Kids and Families.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
KC and RM designed the study, AM and BK collected the data, all authors analysed the data and contributed to the development of the Major Trauma Case Review Tool. All authors drafted/critically revised the article for important intellectual content. All authors read and approved the submitted version.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Ethics approval was obtained from NSW Population and Health Services Research Ethics Committee (HREC/15/CIPHS/6).
About this article
Cite this article
Curtis, K., Mitchell, R., McCarthy, A. et al. Development of the major trauma case review tool. Scand J Trauma Resusc Emerg Med 25, 20 (2017). https://doi.org/10.1186/s13049-017-0353-5