Implementation of checklists in health care; learning from high-reliability organisations
© Thomassen et al; licensee BioMed Central Ltd. 2011
Received: 12 July 2011
Accepted: 3 October 2011
Published: 3 October 2011
Checklists are common in some medical fields, including surgery, intensive care and emergency medicine. They can be an effective tool to improve care processes and reduce mortality and morbidity. Despite the seemingly rapid acceptance and dissemination of the checklist, there are few studies describing the actual process of developing and implementing such tools in health care. The aim of this study is to explore the experiences from checklist development and implementation in a group of non-medical, high reliability organisations (HROs).
A qualitative study based on key informant interviews and field visits followed by a Delphi approach. Eight informants, each with 10-30 years of checklist experience, were recruited from six different HROs.
The interviews generated 84 assertions and recommendations for checklist implementation. To achieve checklist acceptance and compliance, there must be a predefined need for which a checklist is considered a well suited solution. The end-users ("sharp-end") are the key stakeholders throughout the development and implementation process. Proximity and ownership must be assured through a thorough and wise process. All informants underlined the importance of short, self-developed, and operationally-suited checklists. Simulation is a valuable and widely used method for training, revision, and validation.
Checklists have been a cornerstone of safety management in HROs for nearly a century, and are becoming increasingly popular in medicine. Acceptance and compliance are crucial for checklist implementation in health care. Experiences from HROs may provide valuable input to checklist implementation in healthcare.
It has been said that while medicine used to be inefficient, simple, and safe it is now effective, highly complex, and dangerous . Adverse events are documented to affect more than one in 25 hospital patients . Failure to check equipment and lack of vigilance are examples of factors associated with adverse events [3, 4]. Realising how prone we as humans are for short term memory loss, it is striking how many potentially dangerous medical procedures are based on perfect memory . In this context, it is rather strange that checklists are not used more often in medicine.
Checklists are common in some medical fields, and can be an effective tool to improve care processes and reduce mortality and morbidity [6–11]. Checklists are also shown to be a useful tool in pre-hospital and emergency medicine [12, 13]. However, the development and implementation of medical checklists share some of the same challenges as other quality improvement work in medicine, and can be a complex and resource-demanding exercise. Despite the seemingly rapid acceptance and dissemination of the WHO Safe Surgery Checklist, there are few studies describing the actual process of developing and implementing such tools in health care [14, 15].
High reliability organisations (HROs) are often referred to in the medical literature because they perform hazardous and complex operations with an exceptionally low failure rate [16, 17]. Examples of HRO are aviation and aerospace industry, nuclear power production, fire fighting, military operations, and engineering.
Characteristics of HROs
Preoccupation with the possibility of failure
Resistance to oversimplification
Sensitivity to "sharp-end" operations
Commitment to resilience and self-preservation
Deference to shifting locations of expertise
The study was approved by the local Institutional Research Ethics Committee. A triangulation of methods was chosen. In order to gain in-depth and cultural understanding, a qualitative approach with key informant interviews and field visits were used . The results from the interviews were further analysed, by health care workers, using a Delphi process.
Data on the key informants and their organisations
Type of HRO
Key informant's organisational rank & positions
Personal experience with checklist use (years)
Reactor operator & security manager
Statoil ASA, Norway
Norwegian Air Ambulance Service
Captain and flight safety officer
Submarine Training Centre, Norway
Military special operations
Air Force Special Operation Unit, Norway
National Search & Rescue Service, Norway
Captain and deputy executive officer
Interviews and analysis
The key informant interviews generated 84 assertions and recommendations for checklist development and implementation (available from the corresponding author). Seventeen assertions were judged not to be transferrable to health care, and were excluded from further analysis. The assertions were categorized into five main groups with additional subgroups. Citations are in italics and the individual source is identified with capital letters.
Assumptions for checklist acceptance
Several of the informants underlined that a checklist is a tool, not a goal in itself. In order to build a reliable safety culture and achieve checklist acceptance, two basic assumptions must be met; firstly, "There must be a predefined problem that a checklist is the right tool for solving" (C) and secondly, "The end user must not get the feeling that he or she is deprived of the opportunity to apply common sense" (A).
Stakeholders in checklist development
All of the informants expressed that proximity to the checklist development process is very important. One informant cautioned about merely copying checklists; "There must be a built-in scepticism as to whether the experiences of others are relevant for my organisation" (B). Two informants said; "Proximity between the checklist owner and the end user is important" (D) and "Checklists should be made within the organisation that intends to use it" (G).
Stakeholders in the checklist development and implementation process.
"The end-user must be involved in the development process" (F)
"The leader of the development process must be one of the 'sharp-end' users" (D)
Head of department
"The checklist must be developed by the users, but quality assured by the responsible head of the department" (G)
Indirectly affected parties
"If an operation for which a checklist is planned to be developed affects a third party, the third party must also be informed and consulted" (H)
"Get an authority, a leading person, or someone with a good reputation to represent the checklist implementation" (H)
Characteristics of the checklist itself
All informants agreed that limiting the length of a checklist is crucial for its feasibility and usefulness. One informant (E) referred to the resuscitation "ABC checklist". He believed that the reason that this checklist is well known worldwide is because of its simplicity (e.g. it follows the alphabet and is hierarchical, where A is more important than B, and B is more important than C). Another informant warned about the gradual extension of an initially short and well-adapted checklist; "Be careful not to extend checklists because someone - often inexperienced personnel - calls for a more comprehensive list. Too many additions may result in a long and useless checklist" (B).
Lay-out and design
"Be extremely attentive to font and graphics" (E). The informants had experiences with numerous design types. "The design must be adapted to the surroundings, e.g. noise, light conditions, and vibrations" (D). One informant recommended that a professional graphics designer be used to produce the final version of the checklist (G).
According to the informants, none of the HROs used any specific validated methods for selecting the specific content of checklists, but some important issues from this process were mentioned. One informant (B) described the "20/80 rule" (the Pareto-principle). His organisation believed that 20% of the possible adverse events would cause 80% of the consequences. Hence, this principle was used to identify and prioritize the content of their checklists. Several informants emphasized the importance of a precise and operationally-suited checklist. "The checklist must describe exactly the intended operation" (A) and "The checklist, and its content, must directly reflect the specific operation and its surroundings" (E).
Human factors during checklist utilisation
Communication and understanding during checklist performance.
"The end user must be familiar with the background of each point in the checklist" (C)
Terminology and communication
"It is important with a predefined and agreed upon phraseology - closed loop communication - " (F)
"The language must be adapted to the present terminology within the organisation " (H)
Understanding and situational awareness
"The understanding of the text must be unambiguous or clearly defined" (H)
"Be aware of automation, i.e. performing the checklist without being mentally present" (A)
Revisions and validation
Methods and approaches for checklist revision and validation
"Simulation is a good method for validation and testing the usefulness and feasibility of checklists" (C)
"Use simulation to practise checklist utilisation" (E)
"Checklists should be validated regularly according to a plan, in a simulator" (G)
"Use the experience from adverse events during debrief to validate and revise checklists" (E)
Feed-back from end-users
"The validating and revision group must listen to the end-users' experiences and frustrations" (D)
"Revision could be done by feedback from the end-user. It is important that the end-users understand that their experiences are wanted and expected, and that such feed-back may have consequences." (B)
Experiences with checklists in health care are limited compared with HROs. Few studies have explored these organisations' acquired knowledge in checklist development and implementation. Our findings suggest several issues worth considering when introducing medical checklists.
Translating ideas from HROs to patient safety
HROs undoubtedly have thorough and trustworthy experience with checklists, but some have questioned whether health care has gone too far in translating ideas from HROs in general . HROs and medicine have many similarities concerning human factors and complex operations. They both fall prey to the limitations of human pathophysiology and the many challenges of man-machine interface. This study is limited to experiences with checklists and will not discuss the many additional issues concerned with building a safety culture.
Acceptance of checklists
Implementing checklists involves many conflicting interests, including the organisational culture and workflow, which are often affected . Oversimplification of potential challenges could easily lead to conflicts between fractions of adopters and opponents. There are several reports of low compliance when checklists are introduced in health care [10, 26]. Our data highlights some important assumptions about achieving checklist acceptance. Firstly, there must be an identified need or problem for which a checklist is considered the right solution. A process is doomed to fail if the safety manager or department head try to force the "sharp-end" personnel to use a checklist to solve an issue that is not recognised as a problem. Secondly, be aware of challenging physicians' self-esteem and historically strong cultural autonomy. Human beings (including physicians) have cognitive limitations such as difficulty in carefully attending to several things simultaneously . An effective checklist will enhance performance during high workload and stressful conditions by freeing mental capacity to perform important tasks in the correct manner and order . However, checklists should not deprive the operator of the opportunity to use common sense or independently make more reasonable decisions.
Proximity and ownership
The participants in our study emphasized that all stakeholders must have ownership, not only for the final checklist but for the entire development process. These views are consistent with the encouragement to make local modifications to the WHO Surgical Safety Checklist . Recent critical reports from the UK call for an urgent nationwide checklist revision [29, 30]. These challenges regarding checklist acceptance may be the result of failed checklist implementation locally, and lack of proximity between the checklist-owner and end-users.
Checklist lay-out and design
The visual elements of a checklist are important and will directly influence its efficacy and feasibility . There are several guidelines for checklist lay-out and design from both medicine and aviation [31–33]. All informants emphasized the importance of short checklists. Long lists should be divided into shorter sections or separate lists. One of the informants from aviation (E) described how they had divided the long "normal procedures checklist" into three chronological sections; "before take off", "in-flight" and "before landing". The WHO safe surgery checklist follows the same logic ("check-in", "time-out", and "sign-out"). Checklists must be easy to find and use, easy to understand, and short.
All but one of the informants had regular and mandatory checklist-training in full scale simulators. Such simulation is an expensive and time consuming training method, but after decades of experience this is still the preferred method (gold standard) for team and checklist training in these HROs. We also believe that team training is the key for successful checklist implementation in health care. Local team training with low cost and simple technical fidelity is a feasible method for rehearsing complex operations . Success in checklist implementation is probably more a matter of will, enthusiasm, and organisational competence than of available time and financial resources .
Flow of expertise and delegation of authority
Health care has traditionally had a strong hierarchical structure. Such a culture can discourage information exchange, resulting in anxiety or unwillingness to provide feedback on a checklist . HROs cultivate a diversity of expertise; hence, authority migrates to the personnel with process-specific knowledge, independent of their hierarchical organisational position . All the informants described a system where the sharp-ends' experiences are regarded as a valuable and essential part of the revision process. Resilience is a key issue in this context, and has been defined as "the intrinsic ability of a system to adjust its functioning prior to, during, or following changes and disturbances." One cornerstone of resilience is to understand and obtain experiences from both the sharp and blunt end . A self-reinforcing and destructive process may develop if a hierarchical structure prevents feedback from important stakeholders during checklist implementation.
Checklists reduce prospective memory failures
Prospective memory (PM) describes the ability to remember to carry out actions (in the near future) that are planned after a delay or interruption . Unpredictability, delays, and interruptions are frequent in health care . During a stressful situation, memory is also likely to be more error-prone . Cognitive aids like checklists are shown to increase performance in health care when solving complex and time-critical tasks [42, 43]. All the informants in our study described a culture where cognitive aids had an essential role. Medicine has traditionally relied on memory as the basis of diagnosis and treatment . It has been argued that "PM failures in medicine must not be forgotten any more" .
Strengths and limitations
The key strength of this study is the diversity of the informants, who provided a broad information source. Despite the nature of their often classified operations, none of those invited to take part in the study declined to participate. This enabled insight from an often inaccessible safety culture. Additionally, each informant had 10-30 years of cultural experience and an understanding of their organisations. Still, the informants' experiences may not have reflected all the relevant issues regarding checklist development and implementation from their own HRO, and the lack of female informants may also have limited the data to some degree.
In this study we have obtained experiences from informants representing six different HROs. It is likely to believe that these informants, together in an expert consensus process, could have developed and provided further understanding in checklist implementation.
Checklists in HROs are highly valued and given resources to develop, use and validate. Critical comments in health care are concerned if checklists may be cost-ineffective . This line of criticism is debated claiming that checklists are a cost saving quality improvement strategy . Studies concerning cost - benefit should be encouraged.
Checklists are one of the cornerstones of the safety culture of HROs, and their main purpose is to free workers' mental capacity to fully attend to operations. Close cooperation with the "sharp-end" and ownership for all stakeholders throughout the entire checklist implementation process are fundamental requirements in HROs. Despite complex and hazardous operations, HROs manage to keep checklists short. Checklists must reflect the processes they are intended to structure. Simulation is an essential method for training in reliable communication, increasing checklist use, and identifying the need for and implementing revisions.
What is already known on this topic
The introduction of checklists into health care has been slow, despite documented effects on morbidity and mortality.
HROs often use checklists, and aviation has been described as the archetype of HROs with extensive checklist experience.
Checklists can be viewed as simple tools, but their implementation is not so simple.
What this study adds
A recognised and predefined problem is a requirement for checklist acceptance.
The end-users ("sharp end" personnel) are the key stakeholders throughout the checklist development process.
Proximity for all stakeholders throughout the entire checklist implementation process is fundamental.
Even for complex and hazardous operations, HROs have managed to keep checklists short.
Simulation is a suitable and necessary part of checklist testing, implementation, and revision
High reliability organisations
World Health Organisation.
Acknowledgements and funding
The authors thank the informants who participated in the study, as well as Professor Torben Wisborg for valuable input during the study period. O.T thanks the Norwegian Air Ambulance Foundation and the members for financial support during the study period.
- Chantler C: The role and education of doctors in the delivery of health care. Lancet. 1999, 353: 1178-1181. 10.1016/S0140-6736(99)01075-2.View ArticlePubMedGoogle Scholar
- Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, Newhouse JP, Weiler PC, Hiatt HH: Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991, 324: 370-376. 10.1056/NEJM199102073240604.View ArticlePubMedGoogle Scholar
- Marcus R: Human factors in pediatric anesthesia incidents. Paediatr Anaesth. 2006, 16: 242-250. 10.1111/j.1460-9592.2005.01771.x.View ArticlePubMedGoogle Scholar
- Cooper JB, Newbower RS, Long CD, McPeek B: Preventable anesthesia mishaps: a study of human factors. 1978. Qual Saf Health Care. 2002, 11: 277-282. 10.1136/qhc.11.3.277.PubMed CentralView ArticlePubMedGoogle Scholar
- Shillito J, Arfanis K, Smith A: Checking in healthcare safety: theoretical basis and practical application. Int J Health Care Qual Assur. 2010, 23: 699-707. 10.1108/09526861011081831.View ArticlePubMedGoogle Scholar
- Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Tayler B, Gawande AA, Safe Surgery Saves Lives study Group: A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009, 360: 491-499. 10.1056/NEJMsa0810119.View ArticlePubMedGoogle Scholar
- Verdaasdonk EG, Stassen LP, Hoffmann WF, van der Elst M, Dankelman J: Can a structured checklist prevent problems with laparoscopic equipment?. Surg Endosc. 2008, 22: 2238-2243. 10.1007/s00464-008-0029-3.View ArticlePubMedGoogle Scholar
- Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, Sexton B, Hyzy R, Welsh R, Roth G, Bander J, Kepros J, Goeschel C: An intervention to decrease catheter related bloodstream infections in the ICU. N Engl J Med. 2006, 355: 2725-2732. 10.1056/NEJMoa061115.View ArticlePubMedGoogle Scholar
- Lingard L, Regehr G, Orser B, Reznick R, Baker GR, Doran D, Espin S, Bohnen J, Whyte S: Evaluation of a preoperative checklist and team briefing among surgeons, nurses, and anesthesiologists to reduce failures in communication. Arch Surg. 2008, 143: 12-17. 10.1001/archsurg.2007.21. discussion 18View ArticlePubMedGoogle Scholar
- Thomassen O, Brattebo G, Softeland E, Lossius HM, Heltne JK: The effect of a simple checklist on frequent pre-induction deficiencies. Acta Anaesthesiol Scand. 2010, 54: 1179-1184. 10.1111/j.1399-6576.2010.02302.x.View ArticlePubMedGoogle Scholar
- de Vries EN, Prins HA, Crolla RM, den Outer AJ, van Andel G, van Helden SH, Schlack WS, van Putten MA, Gouma DJ, Dijkgraaf MG, Smorenburg SM, Boemeester MA, SURPASS Collaborative Group: Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010, 363: 1928-1937. 10.1056/NEJMsa0911535.View ArticlePubMedGoogle Scholar
- Mayo PH, Hegde A, Eisen LA, Kory P, Doelken P: A program to improve the quality of emergency endotracheal intubation. J Intensive Care Med. 2011, 26: 50-56. 10.1177/0885066610384070.View ArticlePubMedGoogle Scholar
- Mackenzie R, French J, Lewis S, Steel A: A pre-hospital emergency anaesthesia pre procedure checklist. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 2009, 17 (Suppl 3): O26-10.1186/1757-7241-17-S3-O26.PubMed CentralView ArticleGoogle Scholar
- Hales B, Terblanche M, Fowler R, Sibbald W: Development of medical checklists for improved quality of patient care. Int J Qual Health Care. 2008, 20: 22-30.View ArticlePubMedGoogle Scholar
- Winters BD, Gurses AP, Lehmann H, Sexton JB, Rampersad CJ, Pronovost PJ: Clinical review: checklists - translating evidence into practice. Crit Care. 2009, 13: 210-PubMed CentralView ArticlePubMedGoogle Scholar
- Hales BM, Pronovost PJ: The checklist--a tool for error management and performance improvement. J Crit Care. 2006, 21: 231-235. 10.1016/j.jcrc.2006.06.002.View ArticlePubMedGoogle Scholar
- Weick K, Kathleen M: Managing the Unexpected-Assuring High Performance in an Age of Complexity. 2001, Sna Francisco, CA, USA: Jossey-BassGoogle Scholar
- Morath J: To do no harm. 2005, San Francisco: Jossey-BassGoogle Scholar
- Crabtree BF, Miller W: Doing Qualitative Research. 1999, Thousands Oaks: Sage Publications, Inc, 2Google Scholar
- Philibert I: Use of strategies from high-reliability organisations to the patient hand off by resident physicians: practical implications. Qual Saf Health Care. 2009, 18: 261-266. 10.1136/qshc.2008.031609.View ArticlePubMedGoogle Scholar
- Kohn L, Corrigan J, Donaldson M: To Err is Human. Building a Safer Health System. 2000, Washington, D.C: National Academy PressGoogle Scholar
- Jones J, Hunter D: Consensus methods for medical and health services research. Bmj. 1995, 311: 376-380.PubMed CentralView ArticlePubMedGoogle Scholar
- Powell C: The Delphi technique: myths and realities. J Adv Nurs. 2003, 41: 376-382. 10.1046/j.1365-2648.2003.02537.x.View ArticlePubMedGoogle Scholar
- Rogers J: Have we gone too far in translating ideas from aviation to patient safety? Yes. BMJ. 2011, 342: c7309-10.1136/bmj.c7309.View ArticleGoogle Scholar
- Laurance J: Peter Pronovost: champion of checklists in critical care. Lancet. 2009, 374: 443-10.1016/S0140-6736(09)61439-2.View ArticlePubMedGoogle Scholar
- Vats A, Vincent CA, Nagpal K, Davies RW, Darzi A, Moorthy K: Practical challenges of introducing WHO surgical checklist: UK pilot experience. BMJ. 2010, 340: b5433-10.1136/bmj.b5433.View ArticlePubMedGoogle Scholar
- Degani A, Wiener E: Human Factors of Flight-Deck Checklists: The normal Checklist. 1990, Moffet Field, California: NASAGoogle Scholar
- WHO: The WHO Surgical Safety Checklist: Adaptation Guide. 2009, World Health Organization, GenevaGoogle Scholar
- Hancorn K, Blair S: Checklist culture. WHO needs changing. BMJ. 2010, 340: c909-10.1136/bmj.c909.View ArticlePubMedGoogle Scholar
- Mahaffey PJ: Checklist culture. Seductions of the WHO safe surgery checklist. BMJ. 2010, 340: c915-10.1136/bmj.c915.View ArticlePubMedGoogle Scholar
- Verdaasdonk EG, Stassen LP, Widhiasmara PP, Dankelman J: Requirements for the design and implementation of checklists for surgical processes. Surg Endosc. 2008Google Scholar
- Cival Aviation Authority C: Guidence on the Design, Presentation and Use of Emergency and Abnormal Checklists. 2006, Civil Aviation AuthorityGoogle Scholar
- Degani A: Cocpit checklists: Concept, design and use. Human Factors. 1993, 35: 28-43.Google Scholar
- Wisborg T, Brattebo G, Brinchmann-Hansen A, Uggen PE, Hansen KS: Effects of nationwide training of multiprofessional trauma teams in norwegian hospitals. J Trauma. 2008, 64: 1613-1618. 10.1097/TA.0b013e31812eed68.View ArticlePubMedGoogle Scholar
- Wisborg T, Brattebo G: Keeping the spirit high: why trauma team training is (sometimes) implemented. Acta Anaesthesiol Scand. 2008, 52: 437-441. 10.1111/j.1399-6576.2007.01539.x.View ArticlePubMedGoogle Scholar
- Gillespie BM, Chaboyer W, Wallis M, Fenwick C: Why isn't 'time out' being implemented? An exploratory study. Qual Saf Health Care. 19 (2): 103-106.
- Nemeth C, Wears R, Woods D, Hollnagel E, Cook R: Minding the Gaps: Creating Resilience in Health Care Performance and Tools. 2008Google Scholar
- Hollnagel E: Resilience Engineering in Practice. 2011, Surrey: Ashgate Publishing LimitedGoogle Scholar
- Dieckmann P, Reddersen S, Wehner T, Rall M: Prospective memory failures as an unexplored threat to patient safety: results from a pilot study using patient simulators to investigate the missed execution of intentions. Ergonomics. 2006, 49: 526-543. 10.1080/00140130600568782.View ArticlePubMedGoogle Scholar
- Chisholm CD, Collison EK, Nelson DR, Cordell WH: Emergency department workplace interruptions: are emergency physicians "interrupt-driven" and "multitasking"?. Acad Emerg Med. 2000, 7: 1239-1243. 10.1111/j.1553-2712.2000.tb00469.x.View ArticlePubMedGoogle Scholar
- Kuhlmann S, Piel M, Wolf OT: Impaired memory retrieval after psychosocial stress in healthy young men. J Neurosci. 2005, 25: 2977-2982. 10.1523/JNEUROSCI.5139-04.2005.View ArticlePubMedGoogle Scholar
- Harrison TK, Manser T, Howard SK, Gaba DM: Use of cognitive aids in a simulated anesthetic crisis. Anesth Analg. 2006, 103: 551-556. 10.1213/01.ane.0000229718.02478.c4.View ArticlePubMedGoogle Scholar
- Hart EM, Owen H: Errors and omissions in anesthesia: a pilot study using a pilot's checklist. Anesth Analg. 2005, 101: 246-250. 10.1213/01.ANE.0000156567.24800.0B. table of contentsView ArticlePubMedGoogle Scholar
- Sanders RD, Jameson SS: A surgical safety checklist. N Engl J Med. 2009, 360: 2373-author reply 2374-2375PubMedGoogle Scholar
- Semel ME, Resch S, Haynes AB, Funk LM, Bader A, Berry WR, Weiser TG, Gawande AA: Adopting a surgical safety checklist could save money and improve the quality of care in U.S. hospitals. Health Aff (Millwood). 2010, 29: 1593-1599. 10.1377/hlthaff.2009.0709.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.