- Original research
Risk assessment of pre-hospital trauma airway management by anaesthesiologists using the predictive Bayesian approach
Scandinavian Journal of Trauma, Resuscitation and Emergency Medicinevolume 18, Article number: 22 (2010)
Endotracheal intubation (ETI) has been considered an essential part of pre-hospital advanced life support. Pre-hospital ETI, however, is a complex intervention also for airway specialist like anaesthesiologists working as pre-hospital emergency physicians. We therefore wanted to investigate the quality of pre-hospital airway management by anaesthesiologists in severely traumatised patients and identify possible areas for improvement.
We performed a risk assessment according to the predictive Bayesian approach, in a typical anaesthesiologist-manned Norwegian helicopter emergency medical service (HEMS). The main focus of the risk assessment was the event where a patient arrives in the emergency department without ETI despite a pre-hospital indication for it.
In the risk assessment, we assigned a high probability (29%) for the event assessed, that a patient arrives without ETI despite a pre-hospital indication. However, several uncertainty factors in the risk assessment were identified related to data quality, indications for use of ETI, patient outcome and need for special training of ETI providers.
Our risk assessment indicated a high probability for trauma patients with an indication for pre-hospital ETI not receiving it in the studied HEMS. The uncertainty factors identified in the assessment should be further investigated to better understand the problem assessed and consequences for the patients. Better quality of pre-hospital airway management data could contribute to a reduction of these uncertainties.
Pre-hospital endotracheal intubation (ETI) has been considered the gold standard for airway protection and to ensure oxygenation and controlled ventilation in severely injured patients [1–3]. Despite this, studies on the clinical impact of pre-hospital ETI are divergent in their conclusions. Some studies indicate an increased survival related to pre-hospital ETI [4, 5], whereas others indicate the opposite [6–8]. Several authors have claimed that pre-hospital ETI is associated with poor quality and high rates of complications that are more likely to kill than to save the patient [9–13]. Securing the airway by ETI represents a complex intervention  consisting of several critical factors and events. The poor quality and adverse events may be linked to choice of procedure (with or without drugs); lack of provider experience, training and exposure; or insecure and complicated treatment environments [15–17]. Despite high success rates with ETI , even airway experts like anaesthesiologists in emergency medical services (EMS) may face challenges when managing the airway of traumatised patients outside the hospital [19–21].
These challenges are closely related to quality of care and need to be addressed and investigated beyond counting complications and success rates.
Risk assessment methods are useful to investigate complex systems and provide insight into risks, but also to identify factors that influence risk to guide risk reducing measures and improve quality . Many regard predictive risk assessments as especially well suited to health care issues because of their ability to include human factors in the assessment [22–24]. The novel predictive Bayesian approach in particular has been advocated for this use [22, 25]. Still, the experience with the use of such risk assessment methods in health care is limited. The predictive Bayesian approach focuses on observable quantities: the actual population and the knowledge available [22, 25]. Because this approach avoids the use of fictional parameters, it is regarded as a simple predictive risk assessment that is well suited for the study of health care issues [22, 25].
The aim of this study was to investigate the quality of care in pre-hospital airway management by anaesthesiologists in patients with severe trauma and to identify possible areas for improvement. Specifically, we assessed the risk of a trauma patient not receiving pre-hospital ETI when there was an indication for ETI. The risk assessment was performed in a typical Norwegian HEMS.
The risk assessment was performed according to the principles of the predictive Bayesian approach [22, 25, 26]. Initially, we defined the adverse event of interest as "Patient arriving in the emergency department (ED) with an unsecured airway when the indication to secure it was given pre-hospital", hereafter referred to as the "top event". The focus of the risk assessment was to determine the probability of the event and its consequences within the system analysed, to reflect on the uncertainties involved and to analyse the process leading to the event and its consequences.
To analyse the causation leading to the event of interest, we constructed a fault tree. Fault trees are logical descriptions of the cumulative effects of faults within a system that show cause and effect relations among basic or initiating events that culminate in the adverse event of interest [24, 27].
Risk influencing factors
We then incorporated risk influencing factors into the risk description to include human and organisational factors in the assessment. The risk influencing factors were defined by three of the authors (SS, HML and AN), all experienced HEMS physicians. We also gave the risk influencing factors a score to reflect their quality and performance in the system assessed. To simplify, we only used three score levels: good, average and poor (Table 1). This simplification is in line with previous work .
To calculate the probability of the top event in the analysed system based on the risk influencing factors, we first agreed on a set of probabilities for the basic events in a generic system. We assigned probabilities for three cases: all risk influencing factors poor, all risk influencing factors average and all risk influencing factors good. Based on these values, we then calculated the appropriate "adjustment factor" needed in the analysis when the status and weighted influence of the risk influencing factors were taken into consideration. For this calculation, we used the approach described by Aven , adjusted for our purposes with only three score levels.
The risk influencing factors were then assigned their appropriate scores within the system assessed by our expert judgement, and the probability of the initiating event was computed, given the assigned quality and performance of the risk influencing factors.
The knowledge basis used for the assessment is found in Table 2.
To analyse the consequences of the event, we constructed an event tree. Event trees are logical structures that map out the possible consequences of the event of interest and the pathways leading up to the consequences . It is important to notice that event trees are based on the assessors' interpretation of the causations leading up to the possible outcomes. This implies that the assessor has to make certain assumptions that need to be declared when the event tree is presented. Two important assumptions were made in our case. Firstly, that time is an important factor, meaning that delay in treatment impacts outcome. This was however not visualized in the fault tree. Secondly, the clinical state of the patient following the traumatic injury was not taken into consideration when estimating the outcomes, as this would make the assessment to complex.
The possible consequences of the initiating event were described based on the knowledge basis, our knowledge of the system receiving the patients in the ED and the risk analysis and are presented in a risk matrix, where we defined four main categories with five different frequencies/probability categories.
Uncertainty assessment and sensitivity analysis
In the uncertainty assessment, we identified uncertainty factors that were judged to have an influence on risk. Uncertainty factors are related to the interpretation of phenomena or assumptions made in the risk assessment (based on the knowledge available) that can turn out to be wrong in the future when better knowledge of the phenomena is available . Their importance with respect to the effect on risk was categorised as minor, moderate or major based on the description by Flage and Aven . This categorisation was case-specific for the system assessed and was based on the judgement of the assessor. To further investigate the importance of these uncertainty factors, we performed a sensitivity analysis where we investigated how changing the assumptions or interpretations made in the risk analysis would influence the result of the risk analysis. If changing them was found to be important for the risk indices (probabilities of adverse events and consequences) under consideration, it was assigned a high sensitivity; conversely, it was assigned a low sensitivity if changes had little or no influence on the risk indices . Uncertainty factors subject to large uncertainties that would also change the risk indices by even a small amount were regarded as having a significant effect on risk. These uncertainty factors should be subjected to further investigation to increase the knowledge of the phenomena.
In this study we identified four basic events leading up to the main event, thus constructing a fault tree as shown in Figure 1. Based on our evaluation, we decided that all risk influencing factors influenced all basic events, but that the weighted influence was different for each basic event. Figure 2 shows a simple influence diagram demonstrating the influence of the risk influencing factors on the basic events. Based on our knowledge and judgement of the system analysed we assigned the following scores for the risk influencing factors:
Culture and attitudes: average
Providers' experience and knowledge: average
Protocol compliance: poor
With the assigned risk influencing factor basis, we calculated the probability of the top event at 29%, meaning that we would expect the event to take place in 29 of 100 cases if we were to observe the system today. The event tree constructed for the consequence analysis is presented in Figure 3. In the consequence analysis we found the probability of "no harm" and "possible sequela with prolonged hospital stay" to be almost equal (Figure 4) given the event where the patient arrives in the ED without a secured airway when the indication for ETI was given pre-hospital. The consequence "possible sequelae with a prolonged hospital stay" was assigned a probability of at least 10-50% during one year, but less than 1-10 incidents occurring during one year (Figure 4). The probability of death following the event was assigned as less than 1% during one year (Figure 4).
Uncertainty factors identified in the uncertainty assessment are listed in Table 3. In the sensitivity analysis, we found none of the uncertainty factors to have significant sensitivity, but all factors regarding the system and culture and attitudes were assigned a potentially moderate significance, meaning that changing the factors would somewhat change the risk indices too.
In our risk assessment of the typical Norwegian anaesthesiologist-manned HEMS, we found that the probability of a patient arriving in the hospital without ETI when there was a pre-hospital indication for it was as high as 29%.
Most studies on pre-hospital advanced airway management report on complication- or success rates and are either retrospective [30, 31] or prospective observational studies [12, 32–35]. In both cases they only present the incidence of complications or successful ETI in the system where the data was collected at the time of collection. The transferability of the results to other systems or even to the same system may be questionable as other factors influence practice in other systems, and systems may change. Predictive risk assessments like the predictive Bayesian approach may provide a more correct picture of the problem today because it uses a broader knowledge basis (qualitative and quantitative data, and expert judgement) to assess the problem - or rather the risk - as it is today [22, 25, 26]. It is also tailored for the system being assessed in that it uses system specific or -relevant knowledge in the risk assessment . This means that the risk indices assigned in the risk assessment of our top event expresses what we expect the risk to be today in the system assessed.
The predictive Bayesian approach is not only useful for describing a risk picture, it also provides us with a tool to visualise and analyse the causes leading up to the event of interest and to identify where improvements are needed and would have the greatest effect. From a patient safety perspective, this is essential: we often know there is a problem, but we do not have a sufficient understanding of the mechanisms causing the problem. Besides the initiating events in the fault tree, the risk influencing factors provide a good indication of where measures to influence the risk have impact. By changing the values of the assigned probabilities for the basic events in the fault tree, we can for example get an impression of where risk-reducing measures have the greatest impact. If all risk influencing factors in our case were scored as good, the probability of the top event could be reduced to 3%, or with all risk influencing factors scored as average, to 17%. Changing the system and culture and attitudes risk influencing factors to a level where they would be scored as good would change the assigned probability of the top event to 14%, whereas optimising only the providers' competence risk influencing factor to the level where it would be scored as good would lead to an assigned probability of 24%. This implies that risk-reducing measures in our system should focus on system and culture and attitudes.
To establish which aspects of these risk influencing factors that need improvement, we think a key task would be to establish why the physician in charge decides to refrain from securing the airway pre-hospital. In our audit of the HEMS studied (unpublished data, manuscript in preparation), which was part of the knowledge basis for the risk assessment, some physicians have commented that they abstained from pre-hospital ETI because of a short transport distance. The same data show that transport time was lower in the group that was intubated in the ED compared to those intubated pre-hospital. Further, the audit data indicate that the patients intubated pre-hospital were more severely injured than those intubated in the ED because of lower RTSs and GCSs. This might explaining why ETI was postponed in some cases: short transport distance and less serious injury. Still, the patients were intubated immediately upon arrival in the ED, indicating that there was a need to secure the airway. This raises the question of whether the postponed ETI had any impact on the patients' condition. A recent study from the Netherlands  showed a failure to adhere to guidelines for pre-hospital ETI in traumatic brain injury in almost half of the studied population. Further, the authors found a negative influence on respiratory and metabolic parameters in patients not intubated . Other studies have also shown that poor oxygenation and ventilation may worsen outcome [37–39]. If we assert that hypoxia and hypoventilation is more likely to occur in a non-intubated patient, then to abstain from ETI to reduce scene to door time has the potential to harm the patient. The findings of a recent study also indicate that delayed treatment of critically injured patients until arrival in the trauma centre worsens outcome . The potential effect of delayed treatment, in our case ETI, was regarded as important in the consequence analysis of our top event and was one of the reasons why we assigned a high probability of possible sequela and delayed hospital stay following the top event. Because we did not take the patients condition following the injury into account in the consequence analysis, we assigned the impact of the patient's condition on consequences to be an uncertainty factor with major effect on risk.
As the uncertainty analysis revealed there are other important uncertainty factors that influence the analysis at this point. Because we have little knowledge about why the physicians decided not to intubate in some cases the criteria used for deciding to intubate or not was assigned as an uncertainty factor. The quality of the audit data used in the assessment was also difficult to assess. For the purpose of the risk assessment we assumed that the quality of the data recorded in the patient charts would be good, but observed that the completeness of the records was variable. The reliability of the data was therefore assigned to have major effect on risk. Reliable and better quality data would obviously reduce both these uncertainties. For this purpose initiatives like the recent Utstein style template for registering data following pre-hospital airway management are important .
This risk assessment was performed for the Stavanger HEMS based on our knowledge of and experience with the system. The probabilities were assigned by the assessor as an expression of the assessor's uncertainty about the occurrence of the top event [22, 25, 26]. These probabilities are not estimates or predictions of some true value that is thought to exist, as they are often presented in classical risk assessment [22, 25, 26]. As such, they are subjective and conditional on the knowledge available about the system being assessed and the assessor's interpretation of this knowledge. Compared to traditional medical science, a risk assessment conducted according to the predictive Bayesian approach might seem like an exercise in subjective interpretation of data that is prone to assessor bias. Admittedly, this kind of risk assessment is heavily based on the assessors' own judgement, but the assessment process is methodical and traceable. The risk assessment does not attempt to present "truths" or "evidence", but rather focuses on what we can interpret from the knowledge available and what limitations, or rather what uncertainties, there are in the assessment. It is therefore important to declare and assess the uncertainty factors in the analysis. Triangulation is a term used in land surveying to determine the position of an object; by drawing a line to the object from at least three different observation points, the object's exact position can be determined. The same term is used to describe the method of analysing a problem using different sources of information and data and from different viewpoints in qualitative research to develop an overall interpretation  and is also, in our opinion, a valid analogy for assessments like the predictive Bayesian approach. In our case the knowledge basis illustrates the points of triangulation. We believe this to be the most important contribution of the predictive Bayesian approach to risk management in general and in particular to risk management in health care.
Although the risk assessment was limited to one specific Norwegian HEMS, we think the concerns raised may apply to other similar systems. Providing the best quality of care to seriously injured patients can be challenging and as our risk assessment indicates, the decision on when to perform ETI or not may be influenced by many factors. The predictive Bayesian approach provides a tool to perform the same risk assessment in other HEMS systems, provided the appropriate background knowledge from the specific system studied is used.
Our risk assessment of a typical Norwegian HEMS indicated a high probability of the event in which a severely traumatised patient arrives in the hospital without ETI despite a pre-hospital indication for it. The consequence analysis also indicates a high probability of sequela for the patient following this event. There are, however, important uncertainty factors related to this assessment that need to be further investigated to improve our understanding of the event studied. We think better quality of pre-hospital airway management data would contribute to reduce these uncertainties.
Advanced trauma life support for doctors ATLS: manuals for coordinators and faculty. 2008, Chicago, IL: American College of Surgeons, Eight
Smith CE, Walls RM, Lockey D, Kuhnigk H: Advanced Airway Management and Use of Anesthetic Drugs. Prehospital Trauma Care. Edited by: Søreide E, Grande CM. 2001, New York: Marcel Dekker, 203-253.
Walls RM, Murphy MF: Manual of emergency airway management. 2008, Philadelphia, PA: Lippincott Williams & Wilkins, 3
Klemen P, Grmec S: Effect of pre-hospital advanced life support with rapid sequence intubation on outcome of severe traumatic brain injury. Acta Anaesthesiol Scand. 2006, 50: 1250-1254. 10.1111/j.1399-6576.2006.01039.x.
Winchell RJ, Hoyt DB: Endotracheal intubation in the field improves survival in patients with severe head injury. Trauma Research and Education Foundation of San Diego. Arch Surg. 1997, 132: 592-597.
Davis DP, Peay J, Sise MJ, Vilke GM, Kennedy F, Eastman AB, Velky T, Hoyt DB: The impact of prehospital endotracheal intubation on outcome in moderate to severe traumatic brain injury. J Trauma. 2005, 58: 933-939. 10.1097/01.TA.0000162731.53812.58.
Murray JA, Demetriades D, Berne TV, Stratton SJ, Cryer HG, Bongard F, Fleming A, Gaspard D: Prehospital intubation in patients with severe head injury. J Trauma. 2000, 49: 1065-1070. 10.1097/00005373-200012000-00015.
Wang HE, Peitzman AB, Cassidy LD, Adelson PD, Yealy DM: Out-of-hospital endotracheal intubation and outcome after traumatic brain injury. Ann Emerg Med. 2004, 44: 439-450. 10.1016/j.annemergmed.2004.04.008.
Dunford JV, Davis DP, Ochs M, Doney M, Hoyt DB: Incidence of transient hypoxia and pulse rate reactivity during paramedic rapid sequence intubation. Ann Emerg Med. 2003, 42: 721-728. 10.1016/S0196-0644(03)00660-7.
Katz SH, Falk JL: Misplaced endotracheal tubes by paramedics in an urban emergency medical services system. Ann Emerg Med. 2001, 37: 32-37. 10.1067/mem.2001.112098.
Mort TC: Emergency tracheal intubation: complications associated with repeated laryngoscopic attempts. Anesth Analg. 2004, 99: 607-613. 10.1213/01.ANE.0000122825.04923.15. table of contents
Timmermann A, Russo SG, Eich C, Roessler M, Braun U, Rosenblatt WH, Quintel M: The out-of-hospital esophageal and endobronchial intubations performed by emergency physicians. Anesth Analg. 2007, 104: 619-623. 10.1213/01.ane.0000253523.80050.e9.
Wirtz DD, Ortiz C, Newman DH, Zhitomirsky I: Unrecognized misplacement of endotracheal tubes by ground prehospital providers. Prehosp Emerg Care. 2007, 11: 213-218. 10.1080/10903120701205935.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008, 337: a1655-10.1136/bmj.a1655.
Gausche-Hill M: Ensuring quality in prehospital airway management. Curr Opin Anaesthesiol. 2003, 16: 173-181. 10.1097/00001503-200304000-00011.
Mizelle HL, Rothrock SG, Silvestri S, Pagane J: Preventable morbidity and mortality from prehospital paralytic assisted intubation: can we expect outcomes comparable to hospital-based practice?. Prehosp Emerg Care. 2002, 6: 472-475. 10.1080/10903120290938184.
Ummenhofer W, Scheidegger D: Role of the physician in prehospital management of trauma: European perspective. Curr Opin Crit Care. 2002, 8: 559-565. 10.1097/00075198-200212000-00013.
Mackay CA, Terris J, Coats TJ: Prehospital rapid sequence induction by emergency physicians: is it safe?. Emerg Med J. 2001, 18: 20-24. 10.1136/emj.18.1.20.
Sollid SJ, Heltne JK, Soreide E, Lossius HM: Pre-hospital advanced airway management by anaesthesiologists: Is there still room for improvement?. Scand J Trauma Resusc Emerg Med. 2008, 16: 2-10.1186/1757-7241-16-2.
Timmermann A, Eich C, Russo SG, Natge U, Brauer A, Rosenblatt WH, Braun U: Prehospital airway management: a prospective evaluation of anaesthesia trained emergency physicians. Resuscitation. 2006, 70: 179-185. 10.1016/j.resuscitation.2006.01.010.
Einav S, Donchin Y, Weissman C, Drenger B: Anesthesiologists on ambulances: where do we stand?. Curr Opin Anaesthesiol. 2003, 16: 585-591. 10.1097/00001503-200312000-00003.
Aven T, Eidesen K: A predictive Bayesian approach to risk analysis in health care. BMC Med Res Methodol. 2007, 7: 38-10.1186/1471-2288-7-38.
Marx DA, Slonim AD: Assessing patient safety risk before the injury occurs: an introduction to sociotechnical probabilistic risk modelling in health care. Qual Saf Health Care. 2003, 12 (Suppl 2): ii33-38.
Wreathall J, Nemeth C: Assessing risk: the role of probabilistic risk assessment (PRA) in patient safety improvement. Qual Saf Health Care. 2004, 13: 206-212. 10.1136/qshc.2003.006056.
Eidesen K, Sollid SJ, Aven T: Risk assessment in critical care medicine: a tool to assess patient safety. Journal of Risk Research. 2009, 12: 1-14. 10.1080/13669870802456914.
Aven T: Risk analysis: assessing uncertainties beyond expected values and probabilities. 2008, Chichester, England; Hoboken, NJ: Wiley
Dhillon BS: Methods for performing human reliability and error analysis in health care. International Journal of Health Care Quality Assurance. 2003, 16: 306-317. 10.1108/09526860310495697.
Aven T, Hauge S, Sklet S, Vinnem JE: Methodology for Incorporating Human and Organizational Factors in Risk Analysis for Offshore Installations. International Journal of Materials & Structural Reliability. 2006, 4: 1-14.
Flage R, Aven T: Expressing and communicating uncertainty in relation to quantitative risk analysis (QRA). Reliability & Risk Analysis: Theory & Applications. 2009, 2: 9-18.
Lecky F, Bryden D, Little R, Tong N, Moulton C: Emergency intubation for acutely ill and injured patients. Cochrane Database Syst Rev. 2008, CD001429-
Slagt C, Zondervan A, Patka P, de Lange JJ: A retrospective analysis of the intubations performed during 5 years of helicopter emergency medical service in Amsterdam. Air Med J. 2004, 23: 36-37. 10.1016/j.amj.2004.06.004.
Adnet F, Jouriles NJ, Le Toumelin P, Hennequin B, Taillandier C, Rayeh F, Couvreur J, Nougiere B, Nadiras P, Ladka A, Fleury M: Survey of out-of-hospital emergency intubations in the French prehospital medical system: a multicenter study. Ann Emerg Med. 1998, 32: 454-460. 10.1016/S0196-0644(98)70175-1.
Wang H, Cook LJ, Chang CC, Yealy D, Lave J: Outcomes after out-of-hospital endotracheal intubation errors. Resuscitation. 2009, 80: 50-55. 10.1016/j.resuscitation.2008.08.016.
Wang HE, Sweeney TA, O'Connor RE, Rubinstein H: Failed prehospital intubations: an analysis of emergency department courses and outcomes. Prehosp Emerg Care. 2001, 5: 134-141. 10.1080/10903120190939995.
Nakstad AR, Heimdal JH, Strand T, Sandberg M: Incidence of desaturation during prehospital rapid sequence intubation in physician-based helicopter emergency service. Am J Emerg Med. 2010,
Franschman G, Peerdeman SM, Greuters S, Vieveen J, Brinkman AC, Christiaans HM, Toor EJ, Jukema GN, Loer SA, Boer C: Prehospital endotracheal intubation in patients with severe traumatic brain injury: guidelines versus reality. Resuscitation. 2009, 80: 1147-1151. 10.1016/j.resuscitation.2009.06.029.
Davis DP, Dunford JV, Poste JC, Ochs M, Holbrook T, Fortlage D, Size MJ, Kennedy F, Hoyt DB: The impact of hypoxia and hyperventilation on outcome after paramedic rapid sequence intubation of severely head-injured patients. J Trauma. 2004, 57: 1-8. 10.1097/01.TA.0000135503.71684.C8. discussion 8-10
Davis DP, Idris AH, Sise MJ, Kennedy F, Eastman AB, Velky T, Vilke GM, Hoyt DB: Early ventilation and outcome in patients with moderate to severe traumatic brain injury. Crit Care Med. 2006, 34: 1202-1208. 10.1097/01.CCM.0000208359.74623.1C.
Helm M, Hauke J, Lampl L: A prospective study of the quality of pre-hospital emergency ventilation in patients with severe head injury. Br J Anaesth. 2002, 88: 345-349. 10.1093/bja/88.3.345.
Gomes E, Araujo R, Carneiro A, Dias C, Costa-Pereira A, Lecky FE: The importance of pre-trauma centre treatment of life-threatening events on the mortality of patients transferred with severe trauma. Resuscitation. 2010
Sollid SJ, Lockey D, Lossius HM, PAAM: A consensus-based template for uniform reporting of data from pre-hospital advanced airway management. Scand J Trauma Resusc Emerg Med. 2009, 17: 58-10.1186/1757-7241-17-58.
Pope C, Mays N: Qualitative research in health care. 1999, London: BMJ books, 2
The research in this paper was financed through the Bjørn Lind research grant of the Laerdal Foundation for Acute Medicine and a research fellowship from the Norwegian Air Ambulance Foundation. We thank Dr. Andreas Krüger for valuable input in the writing of this manuscript.
The authors declare that they have no competing interests.
SS designed the study, collected the data, performed the statistical analysis and the risk analysis and drafted the manuscript. HL helped design the study, contributed to the statistical analysis and the risk analysis and helped draft the manuscript. AN contributed to the risk analysis and helped review the manuscript. TA contributed to the risk analysis and helped review the manuscript. ES helped design the study and collect the data and reviewed the manuscript. All authors read and approved the final manuscript.