Non-technical skills evaluation in the critical care air ambulance environment: introduction of an adapted rating instrument - an observational study
© Myers et al. 2016
Received: 23 November 2015
Accepted: 27 February 2016
Published: 8 March 2016
In the isolated and dynamic health-care setting of critical care air ambulance transport, the quality of clinical care is strongly influenced by non-technical skills such as anticipating, recognising and understanding, decision making, and teamwork. However there are no published reports identifying or applying a non-technical skills framework specific to an intensive care air ambulance setting. The objective of this study was to adapt and evaluate a non-technical skills rating framework for the air ambulance clinical environment.
In the first phase of the project the anaesthetists’ non-technical skills (ANTS) framework was adapted to the air ambulance setting, using data collected directly from clinician groups, published literature, and field observation. In the second phase experienced and inexperienced inter-hospital transport clinicians completed a simulated critical care air transport scenario, and their non-technical skills performance was independently rated by two blinded assessors. Observed and self-rated general clinical performance ratings were also collected. Rank-based statistical tests were used to examine differences in the performance of experienced and inexperienced clinicians, and relationships between different assessment approaches and assessors.
The framework developed during phase one was referred to as an aeromedical non-technical skills framework, or AeroNOTS. During phase two 16 physicians from speciality training programmes in intensive care, emergency medicine and anaesthesia took part in the clinical simulation study. Clinicians with inter-hospital transport experience performed more highly than those without experience, according to both AeroNOTS non-technical skills ratings (p = 0.001) and general performance ratings (p = 0.003). Self-ratings did not distinguish experienced from inexperienced transport clinicians (p = 0.32) and were not strongly associated with either observed general performance (r s = 0.4, p = 0.11) or observed non-technical skills performance (r s = 0.4, p = 0.1).
This study describes a framework which characterises the non-technical skills required by critical care air ambulance clinicians, and distinguishes higher and lower levels of performance.
The AeroNOTS framework could be used to facilitate education and training in non-technical skills for air ambulance clinicians, and further evaluation of this rating system is merited.
KeywordsNon-technical skills Air ambulance Intensive care Patient transport Clinical training
In health care, preventing errors and avoidable adverse events for patients (patient safety) is paramount. High quality clinical performance requires adequate knowledge and technical ability, but also relies on non-technical skills such as the ability to adapt to a rapidly changing clinical situation and to function as part of a team [1, 2]. Non-technical skills can be defined as “the cognitive, social and personal resource skills that complement technical skills and contribute to safe and efficient task performance” . Even though a high degree of technical expertise in important, this alone is not enough to prevent clinician error or mishap. Non-technical skills are more likely, compared to technical skills, to be sensitive to individual human factors such as fatigue and stress . High risk industries with low tolerance for error (such as aviation and the nuclear power industry) were early to recognise the importance of non-technical skills for safety; these industries developed rating frameworks to evaluate crew performance based on observable behaviours [4, 5]. This approach has also been implemented in high-risk health care domains, where behavioural marker systems are increasingly utilised as part of training or assessment of clinical competence [6, 7].
The air ambulance environment is a complex and dynamic health-care setting, where clinicians work with limited resources to provide very advanced levels of care [8–10]. Highly specialised care is centralised in many modern health care systems and critically ill patients are routinely transported large distances to tertiary hospitals to provide timely access to intensive care [11, 12]. Air ambulance transport teams tend to be small and comprise different professional categories such as nurses, emergency medical technicians, and physicians. Challenges for maintaining patient safety in an aviation environment include managing sometimes acutely life-threatening and rapidly evolving medical issues without the support and facilities available in a hospital environment [13, 14]. High noise levels in the cabin may preclude traditional options for clinical surveillance such as auscultation or audible alarms, and make communication challenging. Clinicians cannot always access additional assistance, resources, or expertise, should problems arise or clinical status change while the patient is in transit. In this context, while good technical expertise is certainly required, it may be non-technical factors such as how well clinicians have planned and anticipated, or how quickly they recognise, understand, and make decisions, that most strongly influence eventual outcome. Well-designed training for air ambulance clinicians should aim to prepare them for the recognised risks to patient care during all phases of transfer, and an assessment framework based on non-technical skills would clearly have a high degree of relevance for this purpose. However there are no published reports identifying or applying a non-technical skills framework specific to an intensive care air ambulance setting.
A number of non-technical skills rating frameworks have been developed for health-care domains closely related to the air ambulance setting, including emergency care [6, 15], critical care , and anaesthesia [17–19]. Each of these frameworks is broadly similar, reflecting the generic nature of non-technical skills’ categories such as situational awareness, decision making, and teamwork , however specific skill elements and behavioural descriptors vary according to the clinical requirements of the specific domain . An existing behavioural rating framework can be adapted to another clinical setting using data gathered directly from the new setting . In the aeromedical setting, the well-established Anaesthetists’ Non-Technical Skills (ANTS) framework  is suitable for this purpose [21, 22]. The ANTS system provides a framework for describing the individual non-technical skills of clinicians as well as a tool to guide their assessment within the clinical workplace for anaesthesia . The overall goal of this study was to assess a newly adapted non-technical skills rating system based on the ANTS system but modified for the air ambulance clinical environment. Our hypothesis was that a non-technical skills framework adapted to a critical care air ambulance environment could discriminate between stronger and weaker non-technical skills performances. We aimed to test this with a volunteer cohort of lesser and more experienced intensive care physicians in a challenging air ambulance transfer simulation, where non-technical skills assessors were blinded to clinician experience.
The project was undertaken in two phases. First, a non-technical skills framework was adapted to the critical care air ambulance setting using the ANTS system as the foundation. The adapted framework was referred to as an aeromedical non-technical skills framework, or AeroNOTS. In the second phase the adapted AeroNOTS framework was utilised to evaluate the non-technical skills observed in clinicians working in simulated inter-hospital transport scenarios.
Adaptation of a non-technical skills framework to the critical care air ambulance setting
Evaluation of non-technical skills using clinical simulation: study setting and participants
The observational study took place in the simulation suite of a New Zealand tertiary hospital; it was timed to take place around a training placement changeover with the aim of recruiting a convenience sample of minimally experienced intensive care transport physicians. Experienced intensive care transport physicians were also recruited and assessed. Prior to the simulations, the intensive care unit (ICU) flight service medical director categorised all the participants as either ‘experienced’ or ‘inexperienced’ in ICU inter-hospital transport. All participants provided informed consent and completed an enrolment questionnaire which included details of training, transport and simulation experience.
Following orientation to the simulator all clinicians completed a critical care inter-hospital transport scenario with a highly experienced flight nurse serving as a ‘confederate’ and standardised team member. The scenario took approximately 20 minutes, with an initial phase set in a high-fidelity regional emergency department where the transport physician and flight nurse took over care of a ventilated patient requiring air ambulance transfer to a tertiary hospital ICU in another city. The second phase was set in a low fidelity helicopter fuselage with actual transport equipment (stretcher, ventilator, monitors) and comparable space restrictions, but no aircraft noise or vibration. In the scenario the patient’s condition deteriorated rapidly, and a life-saving intervention was required [Additional file 2]. Following the scenario an observing ICU consultant facilitated a de-brief, which was not recorded.
Assessment of non-technical skills
Clinicians were informed that purpose of the study was to evaluate methods for assessing clinical performance. They were not specifically told that the key focus for that assessment was on non-technical skills. As recommended when undertaking formal assessment of non-technical skills performance, each skill element was initially rated separately, then final ratings were made at the level of the four main skill categories of task management, team working, situation awareness and decision making . Since communication is required to demonstrate skill elements across all categories there was no specific category for communication in the AeroNOTS system, as with the ANTS system . Possible scores for each skill category and element ranged between 1 and 5, where a rating of ‘5’ was “Excellent – extremely good performance which could serve as a model example for others; patient safety enhanced”, down to ‘1’ which was “Poor - absence of behaviour required by the situation; performance endangered or potentially endangered patient safety” (Fig. 2). Category scores were analysed separately giving a score for each category of between 1 and 5, and then added to give a single summed score (providing an overall non-technical skills score between 4 and 20) [25, 28]. Half marks on the scale were not permitted; assessors were instructed to score at the lower level if they felt the performance fell between two levels on the scale [Additional file 3].
The assessments were carried out independently by two observers who viewed video recordings of the scenarios; assessors were blinded to the experience level of the participants.
Assessment of general clinical performance
The AeroNOTS scores from two assessors for each participant were averaged for further analysis within participant groups. A priori assumptions were that non-technical skills ratings for clinicians more experienced in air transports would be higher than for less experienced clinicians, and that general clinical performance levels would correlate with non-technical skills performance levels. Statistical analysis was undertaken using SPSS software (IBM SPSS Statistics for Windows, Version 22.0. Armonk, New York). Demographic variables were compared using t-tests or Fisher exact tests. AeroNOTS ratings scores (summed total scores between 4 and 20, and individual category scores between 1 and 5) were treated as ordinal data and analysed using rank-based methods. These included Mann–Whitney U to test for differences in performance between groups, Spearman’s rank correlation to examine relationships between different assessment approaches and assessors, and Wilcoxon signed-rank to examine individual scoring from the two assessors. Spearman’s rank correlation was also used to test for association between the immediate general rating of clinical performance compared to non-technical skills rating. The level of statistical significance was set at p < 0.05.
Ethical approval was provided by the University of Otago (Health) Human Ethics Committee, New Zealand (HD12/233 and HD14/44). All clinicians who participated in the clinical simulation study provided signed informed consent.
Non-technical skills framework adapted for the critical care air ambulance setting
Evaluation of non-technical skills using clinical simulation
Clinical simulation study: baseline characteristics of participants
Age (mean ± SD)
36.1 ± 5.6
29.8 ± 2.1
Male gender, n (%)
4 (50 %)
8 (100 %)
Training level, n (%)
2 (25 %)
0 (0 %)
6 (75 %)
2 (25 %)
5 (62.5 %)
1 (12.5 %)
Specialty training programme, n (%)
ICU and Anaesthetics
1 (12.5 %)
2 (25 %)
0 (0 %)
3 (37.5 %)
Critical and Intensive Care Medicine
3 (37.5 %)
0 (0 %)
3 (37.5 %)
2 (25 %)
1 (12.5 %)
1 (12.5 %)
Number of previous inter-hospital patient transports, median (IQR)
45 (25 – 51.5)
0.5 (0 – 4.5)
Number of previous simulations, median (IQR)
15 (2.25 – 20)
5 (5 – 16)
Sleep in 24 hrs pre-scenario, median hours (IQR)
7.0 (6.6 – 7.9)
Work in 24 hrs pre-scenario, median hours (IQR)
8 (1 – 13.5)
6 (1.75 – 7.75)
Samn-Perelli Fatigue Scorea, median (IQR)
3 (1.25 – 4.75)
2.5 (1.25 – 3.75)
Assessment of non-technical skills
Performance ratings from the two assessors were similar, with a high degree of correlation (r s = 0.65, p = 0.006) and no significant difference in overall scores (AeroNOTS summed scores, Wilcoxon signed-ranks test, p = 0.21). These results stayed consistent across the four individual skill categories (correlation coefficients ranging between r s = 0.54 and 0.76, all significantly greater than 0 at p = 0.05). However for the category of ‘task management’ the scores from one assessor rated higher than the other (Wilcoxon signed-ranks test, p = 0.02). The averages of the 2 assessors’ scores were used for the analyses.
Assessment scores: Non-technical skills and general clinical performance
Non-technical skills (AeroNOTS)a, median (IQR)
16 (15.125 – 17.125)
11.75 (8.75 – 14.25)
Task Managementb, median (IQR)
3.75 (3.5 – 4.375)
2.5 (2.125 – 3)
Teamworkb , median (IQR)
4 (3.625 – 4.5)
3 (2.5 – 3.5)
Situational Awarenessb , median (IQR)
3.75 (3.5 – 4.375)
3 (2.5 – 3.375)
Decision Makingb , median (IQR)
4 (3.625 – 4.375)
2.75 (2.125 – 3)
General clinical performancec , median (IQR)
4 (4 – 4)
2.75 (2 – 3)
Self-rated clinical performancec , median (IQR)
4 (3 – 4)
3.5 (2.125 – 4)
This study describes the development and evaluation of a framework to assess non-technical skills in aeromedical transport. The framework discriminated between more and less experienced clinicians, based on their non-technical skill performance during simulated transfer of a critical patient. Both technical and non-technical skills are needed in tandem for good medical team performance and patient safety in a high risk medical environment, and both improve with good training [29–31]. Necessary elements for improving clinical performance include identifying specific skills directly relevant to performance quality, then measuring or assessing those skills in a standardised manner . The results of this study indicate that by characterising the non-technical skills requirements for clinicians, and distinguishing higher and lower levels of non-technical performance, the prototype AeroNOTS framework could be used to facilitate good education and training in non-technical skills. Published standards (on which training curricula may be based) from Europe , New Zealand , and the US , all mandate crew resource management (CRM) training in areas such as decision making, communications processes, team building and maintenance, workload management, and situation awareness, but they include little specific detail to define these skills. The specific behaviours identified in the AeroNOTS instrument can facilitate identification of specific areas for individuals where further training might be beneficial.
In this study, ‘self-ratings’ of performance were not useful in distinguishing different levels of performance, with inexperienced clinicians tending to over-estimate their performance level. Limitations in clinicians’ ability to self-assess performance have been reported previously , but it is possible that our findings were partly a result of recruiting one group of inexperienced clinicians who lacked appropriate inter-hospital transport experience on which to base their self-assessments. It is also possible that self-ratings are more accurate at the extremes, such as when performance is significantly degraded . Further examination of ‘self-rating’ is warranted in light of the fact that critical care air ambulance clinicians are particularly vulnerable to factors like fatigue , and risk management systems generally rely on clinicians ‘self-identifying’ if their performance is compromised .
There is a paucity of literature and no published skills taxonomy, so collecting additional data from the critical care air ambulance domain to adapt the well-established ANTS system was essential . Anaesthesia is a medical speciality with a leading role in addressing patient safety and taking a human factors approach to training and safety , and while a behavioural rating system cannot simply be applied to another specialty area  non-technical skills are broadly generic . As previous authors report significant overlap in the non-technical skills requirements of intensive care and anaesthesia , it was reasonable to expect similarity between the skills required of intensive care air ambulance clinicians and anaesthesia specialists. Both function in teams of variable professional makeup, and so require frameworks where the fundamental focus is on the non-technical skills of individual clinicians, but encompassing how they function as part of a team.
A measurement system suitable for evaluating the non-technical skills of air ambulance clinicians should provide a true (valid) and consistent (reliable) representation of those skills. Face and content validity for the AeroNOTS system were addressed during development by collecting data directly from the aeromedical transport domain (clinician experts and relevant literature). For example, based on literature an addition to the ‘using authority or assertiveness’ element of Teamwork was: “questions others regardless of seniority when they are unsure the right decision has been made”; and a negative behaviour for the ‘gathering information’ element of Situational Awareness: “does not alter layout of the workplace to improve data visibility or audibility” (ability to hear in aircraft is limited so clinicians need to be able to see monitors to make up for this). From focus groups a recurrent theme emerged that experienced air ambulance clinicians “plan for things to go wrong” and “plan for every eventuality”. They also develop strategies for potential vulnerable points in the transfer, such as communication strategies for working with unknown team members, being assertive in acquiring information when working in unfamiliar surroundings, and ensuring they are being listened to. Results from the clinical simulation study provided support for the construct validity of the AeroNOTS framework as a tool for assessing non-technical skills performance. In accordance with the a priori expectations we set, experienced clinicians received higher scores from blinded assessors than inexperienced clinicians, and AeroNOTS scores were strongly correlated with general performance scores.
One limitation in the study design was that while data from all air ambulance clinician groups were informative concerning the phase one AeroNOTS adaptation process, only physicians participated in the second phase clinical simulation study. It was not possible to recruit comparative groups containing inexperienced flight nurses or air ambulance paramedics locally. Ongoing evaluation should therefore include all clinician groups routinely involved in critical care transfer. Another potential limitation is that much of the initial adaptation work and evaluation for the AeroNOTS framework involved clinicians and services based in New Zealand or Australia and it is possible that roles and responsibilities of air ambulance clinicians, and therefore the required non-technical skills, are not the same in all countries . We aimed to provide that international perspective through assessment of the literature, content evaluation survey and field testing, all of which provided support for validity. However more wide-spread evaluation may be required.
This study was not designed to test the reliability of the AeroNOTS rating system. Based on results from studies of non-technical skills in other domains, some variability between assessor ratings, such as in our findings, was not unexpected [16, 41, 42]. The assessors were trained in non-technical skills concepts and use of the rating system , but no attempt was made to calibrate their ratings before the study. This will require further examination if the AeroNOTS framework is to be used for formal assessment of clinical competence.
An air ambulance non-technical skills framework derived from the ANTS tool is capable of distinguishing good and poor performers in a simulated inter-hospital transport setting. Scores can be highly correlated with observed general performance, and also with the general experience of the clinician. Our findings confirm that self-ratings are not useful for distinguishing between higher and lower levels of performance. This framework could be useful in identifying when specific non-technical factors are likely to break down in the air ambulance environment, and facilitating a more structured approach to training and assessment. The AeroNOTS rating system shows utility and applicability for a critical care air ambulance environment, and further evaluation of this framework is merited.
Aeromedical non-technical skills
Anaesthetists’ non-technical skills
Intensive Care Unit
Crew resource management
This project was supported by funding from the University of Otago, New Zealand and Umeå University, Sweden. The authors would also like to thank the staff of the Simulation & Skills Centre, Wellington Regional Hospital, for their help with data collection.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Gordon M, Darbyshire D, Baker P. Non‐technical skills training to enhance patient safety: a systematic review. Med Educ. 2012;46(11):1042–54.View ArticlePubMedGoogle Scholar
- Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2009;53(2):143–51.View ArticlePubMedGoogle Scholar
- Flin RH, O'Connor P, Crichton M. Safety at the sharp end: A guide to non-technical skills. Farnham, UK: Ashgate; 2008.Google Scholar
- Flin R, O'Connor P, Mearns K. Crew resource management: improving team work in high reliability industries. Team Perform Manag. 2002;8(3/4):68–78.View ArticleGoogle Scholar
- Helmreich RL. Managing human error in aviation. Sci Am. 1997;276(5):62–7.View ArticlePubMedGoogle Scholar
- Flowerdew L, Brown R, Vincent C, Woloshynowych M. Development and validation of a tool to assess emergency physicians' nontechnical skills. Ann Emerg Med. 2012;59(5):376–85.View ArticlePubMedGoogle Scholar
- Kodate N, Ross A, Anderson JE, Flin R. Non-Technical Skills (NTS) for enhancing patient safety: achievements and future directions. Jap J Qual Saf Health Care. 2012;7(4):360–70.Google Scholar
- Flabouris A, Runciman WB, Levings B. Incidents during out-of-hospital patient transportation. Anaesth Intensive Care. 2006;34(2):228–36.PubMedGoogle Scholar
- Freebairn R. Interhospital and emergency transfers in New Zealand. N Z Med J. 2012;125(1351):7–10.PubMedGoogle Scholar
- MacDonald RD, Banks BA, Morrison M. Epidemiology of adverse events in air medical transport. Acad Emerg Med. 2008;15(10):923–31.View ArticlePubMedGoogle Scholar
- Warren J, Fromm RE, Orr RA, Rotello LC, Horst HM. Guidelines for the inter-and intrahospital transport of critically ill patients. Crit Care Med. 2004;32(1):256–62.View ArticlePubMedGoogle Scholar
- Brandstrom H, Winso O, Lindholm L, Haney M. Regional intensive care transports: a prospective analysis of distance, time and cost for road, helicopter and fixed-wing ambulances. Scand J Trauma Resusc Emerg Med. 2014;22(1):36.View ArticlePubMedPubMed CentralGoogle Scholar
- Hearns S, Shirley PJ. Retrieval medicine: a review and guide for UK practitioners. Part 2: safety in patient retrieval systems. Emerg Med J. 2006;23(12):943–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Abrahamsen HB, Sollid SJ, Öhlund LS, Røisli J, Bondevik GT. Simulation-based training and assessment of non-technical skills in the Norwegian Helicopter Emergency Medical Services: a cross-sectional survey. Emerg Med J. 2015;32(8):647–53.PubMedPubMed CentralGoogle Scholar
- Cooper S, Cant R, Porter J, Sellick K, Somers G, Kinsman L, et al. Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM). Resuscitation. 2010;81(4):446–52.View ArticlePubMedGoogle Scholar
- Weller J, Frengley R, Torrie J, Shulruf B, Jolly B, Hopley L, et al. Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams. BMJ Qual Saf. 2011;20(3):216–22.View ArticlePubMedGoogle Scholar
- Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R. Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth. 2003;90(5):580–8.View ArticlePubMedGoogle Scholar
- Lyk-Jensen H, Jepsen R, Spanager L, Dieckmann P, Østergaard D. Assessing Nurse Anaesthetists' Non‐Technical Skills in the operating room. Acta Anaesthesiol Scand. 2014;58(7):794–801.View ArticlePubMedGoogle Scholar
- Rutherford JS, Flin R, Irwin A, McFadyen AK. Evaluation of the prototype Anaesthetic Non-technical Skills for Anaesthetic Practitioners (ANTS-AP) system: A behavioural rating system to assess the non-technical skills used by staff assisting the anaesthetist. Anaesthesia. 2015;70(8):907–14.View ArticlePubMedGoogle Scholar
- Reader T, Flin R, Lauche K, Cuthbertson BH. Non-technical skills in the intensive care unit. Br J Anaesth. 2006;96(5):551–9.View ArticlePubMedGoogle Scholar
- Wax R. The ABC’s of Air Medical Resource Management. In: Fredriksen K, editor. Principles and direction of Air Medical Transport. Salt Lake City: Air Medical Physician Association; 2006.Google Scholar
- Martinsen J. Observation and rating HEMS Crew in Non-Technical Skills, CRM Medical Simulation in Norwegian Air Ambulance. Scand J Trauma Resusc Emerg Med. 2015;23 Suppl 2:A21.View ArticlePubMed CentralGoogle Scholar
- Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anaesthesiol. 2011;25(2):215–27.View ArticlePubMedGoogle Scholar
- Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459–67.View ArticlePubMedGoogle Scholar
- Riem N, Boet S, Bould MD, Tavares W, Naik VN. Do technical skills correlate with non-technical skills in crisis resource management: a simulation study. Br J Anaesth. 2012;109(5):723–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Jepsen RM, Spanager L, Lyk-Jensen HT, Dieckmann P, Østergaard D. Customisation of an instrument to assess anaesthesiologists' non-technical skills. Int J Med Educ. 2015;6:17.View ArticlePubMedPubMed CentralGoogle Scholar
- Flin R, Patey R, Glavin R, Maran N. Anaesthetists' non-technical skills. Br J Anaesth. 2010;105(1):38–44.View ArticlePubMedGoogle Scholar
- Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18(2):104–8.View ArticlePubMedGoogle Scholar
- Aggarwal R, Grantcharov TP, Darzi A. Framework for systematic training and assessment of technical skills. J Am Coll Surg. 2007;204(4):697–705.View ArticlePubMedGoogle Scholar
- Pena G, Altree M, Field J, Sainsbury D, Babidge W, Hewett P, et al. Nontechnical skills training for the operating room: A prospective study using simulation and didactic workshop. Surgery. 2015;158(1):300–9.View ArticlePubMedGoogle Scholar
- Yee B, Naik VN, Joo HS, Savoldelli GL, Chung DY, Houston PL, et al. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesthesiology. 2005;103(2):241–8.View ArticlePubMedGoogle Scholar
- Sevdalis N, Hull L, Birnbach DJ. Improving patient safety in the operating theatre and perioperative care: obstacles, interventions, and priorities for accelerating progress. Br J Anaesth. 2012 Dec;109 Suppl 1:i3–i16.
- European Aeromedical Institute EURAMI. Standards Version (2013) 4.0. www.eurami.org. Accessed July 22 2015.
- Ambulance New Zealand. Air Ambulance/Air Search and Rescue Service Standard Version 2. http://www.ambulancenz.co.nz/standards/ (2013). Accessed July 22 2015.
- Commission for Accreditation of Transport Services. 10th Edition Draft Standards. 2015. http://www.camts.org/. Accessed July 22 2015.
- Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094–102.View ArticlePubMedGoogle Scholar
- Myers JA, Haney MF, Griffiths RF, Pierse NF, Powell DM. Fatigue in air medical clinicians undertaking high-acuity patient transports. Prehosp Emerg Care. 2015;19(1):36–43.View ArticlePubMedGoogle Scholar
- Lerman SE, Eskin E, Flower DJ, George EC, Gerson B, Hartenbaum N, et al. Fatigue Risk Management in the Workplace. J Occup Environ Med. 2012;54(2):231–58.View ArticlePubMedGoogle Scholar
- Gaba DM. Anaesthesiology as a model for patient safety in health care. BMJ. 2000;320(7237):785.View ArticlePubMedPubMed CentralGoogle Scholar
- Wisborg T, Manser T. Assessment of non-technical skills in the operating room–one assessment tool per specialty. Acta Anaesthesiol Scand. 2014;58(7):773–4.View ArticlePubMedGoogle Scholar
- Flowerdew L, Gaunt A, Spedding J, Bhargava A, Brown R, Vincent C, et al. A multicentre observational study to evaluate a new tool to assess emergency physicians' non-technical skills. Emerg Med J. 2013;30(6):437–43.View ArticlePubMedGoogle Scholar
- Yule S, Rowley D, Flin R, Maran N, Youngson G, Duncan J, et al. Experience matters: comparing novice and expert ratings of non‐technical skills using the NOTSS system. ANZ J Surg. 2009;79(3):154–60.View ArticlePubMedGoogle Scholar