Skip to main content

Training and assessment of non-technical skills in Norwegian helicopter emergency services: a cross-sectional and longitudinal study



Deficient non-technical skills (NTS) among providers of critical care in helicopter emergency medical services (HEMS) is a threat to patient and operational safety. Skills can be improved through simulation-based training and assessment. A previous study indicated that physicians underwent less frequent training compared to pilots and HEMS crew members (HCM) and that all professional groups in Norwegian HEMS received limited training in how to cope with fatigue. Since then, training initiatives and a fatigue risk management project has been initiated. Our study aimed to explore if the frequency of simulation-based training and assessment of NTS in Norwegian HEMS has changed since 2011 following these measures.


A cross-sectional web-based survey from October through December 2016, of physicians, HCM and pilots from all civilian Norwegian HEMS-bases reporting the overall extent of simulation-based training and assessment of NTS.


Of 214 invited, 109 responses were eligible for analysis. The frequency of simulation-based training and assessment of NTS has increased significantly for all professional groups in Norwegian HEMS, most prominently for the physicians. For all groups, the frequency of assessment is generally lower than the frequency of training.


Physicians in Norwegian HEMS seem to have adjusted to the NTS training culture of the other crew member groups. This might be a consequence of improved NTS training programs. The use of behavioural marker systems systematically in HEMS should be emphasized.


Pre-hospital critical care and transport of critically ill or injured patients involve a significant risk of adverse events [1]. Studies investigating the factors contributing to critical incidents and adverse events in highly dynamic domains of healthcare, such as emergency medicine, have shown that teamwork plays an important role [2]. Team leadership is a critical skill for emergency medicine physicians directly affecting team performance and the quality of patient care [3, 4]. Poor communication has been found to be a significant factor in adverse events in air ambulance transports [5, 6], but overall, research on the causes of human errors in helicopter emergency medical services (HEMS) is still sparse [7].

Systematic training and assessment of non-technical skills (NTS) in HEMS have received little attention in the past [8, 9]. NTS can be defined as the cognitive and interpersonal skills needed to deliver safe care [10]. Seven generic categories of NTS have been suggested: situation awareness, decision-making, communication, teamwork, leadership, managing stress and coping with fatigue [11].

To document the level of simulation-based training and assessment of non-technical skills in 2011 among crew members of the Norwegian HEMS, Abrahamsen and co-workers performed a cross-sectional survey [8]. The main findings from this study was a lack of simulation-based training and assessment for all professional groups in Norwegian HEMS, that physicians underwent significantly less frequent training and assessment compared to pilots and HEMS Crew Members (HCM), and that all groups received limited training in how to cope with fatigue even though they were on call for extended hours. Since then, the Norwegian Air Ambulance Foundation has implemented a crew training camp concept for the Norwegian HEMS [12], initiated a research project of in situ simulation training during on-call hours with the implementation of weekly simulation training at several HEMS bases in Norway [13], and conducted a fatigue risk management project in Norwegian HEMS.

Our study aimed to explore if the frequency of simulation-based training and assessment of non-technical skills in Norwegian HEMS has changed following the training initiatives mentioned above. Our hypothesis is that the frequency of simulation-based training and assessment of NTS has increased in all the three professional groups.



Since the previous survey, one additional HEMS base has been established in Norway. The 12 HEMS bases all have helicopters staffed with a pilot, a HEMS crew member (HCM) and a physician running 24/7 services. One HEMS base is staffed with an additional flight nurse, but because the number of nurses is low, full anonymity could not be guaranteed and this professional group was not included in the previous study. This also applies to the current survey. All Norwegian HEMS physicians are certified or soon-to-be certified anaesthesiologists and employed by the local health enterprise. HCMs and pilots are employed by one of the two flight operators, Norsk Luftambulanse AS and Lufttransport RW AS.


Eight question categories regarding education and training in NTS were attached to a patient safety climate questionnaire (Additional file 1). Except for a minor adaptation in wording to also fit ground ambulance organization, the questionnaire was identical to the previous survey [8]. Similarly, our study focused on the two question categories reporting the overall extent of simulation-based training (question category I6) and assessment (question category I7) in the previous year on a four-point ordinal scale (0, 1–2, 3–5, > 5 times per year) for each of the seven generic NTS categories. The questionnaire also contained seven background variables relating to the respondents’ work characteristics; work area, geographic location, field of competence, patient contact, work hours, experience in the prehospital area and seniority in position.

Data collection

All physicians, HCMs and pilots working in the civilian Norwegian HEMS were invited to participate in an anonymous, cross-sectional web-based survey (SurveyXact™, Rambøll Management Consulting, Oslo, Norway). A link to the survey was distributed via e-mail and five reminders were sent non-responders. The survey was open from October through December 2016.

Statistical analysis

All answers related to simulation-based training and assessment were dichotomized into “some training/assessment” and “no training/assessment”. To visualize the development in training and assessment, ratios of the percentages from 2015 divided by the corresponding percentages from 2011, were calculated and are presented in bar charts across an ordinal scale. A ratio greater than 1, indicates a positive development in the frequency of training and assessment. To support the visuals, a series of two-sided Fisher’s exact test of the dichotomized items were performed. A p-value less than 0.05 should imply a rejection of the null hypothesis, which was no association between the two groups of interest and level of training and assessment. The freeware R 3.1.3 was used for all calculations and visualization producing the results presented in this paper.

Ethical considerations

The study was approved by the Norwegian Centre for Research Data (Ref. no. 2016/45723) and was exempted from ethical approval by the Regional Committee for Medical and Health Research Western Norway (Ref. no. 2015/2249). The participants received information regarding the purpose of the study and that the questionnaires were to be treated in confidence, and their written consent to participate in the study was given at the start of the survey.


In total, 214 physicians, HCMs and pilots in the Norwegian civilian HEMS were invited to participate in the survey. We received 118 responses, yielding a response rate of 55.1%. Nine responses were excluded due to either missing core data, or because respondents stated search and rescue services (SAR) or fixed wing air ambulance as their main job, giving 109 responses eligible for analysis. Of these, 49% (53) were from physicians, 28% (31) from HCM and 23% (25) from pilots. In 2011, the corresponding distribution among the professional groups was 53, 27 and 20%, respectively (Table 1, Fig. 1).

Table 1 Demographic and professional characteristics of the study populations in 2011 and 2015
Fig. 1
figure 1

Inclusion flow chart

Overall training and assessment of NTS in Norwegian HEMS

When evaluating the results for all personnel in Norwegian HEMS as a whole, the frequency of both simulation-based training and assessment for all NTS categories have increased from 2011 to 2015. By statistical testing, we found that all changes were significant except for simulation-based training in “coping with fatigue” (Table 2).

Table 2 Norwegian HEMS personnel with simulation-based training in and assessment of non-technical skills

Training and assessment for each professional group

Physicians were the professional group with most categories with significant increase in training and assessment from 2011 to 2015. The frequency of simulation-based training of decision-making, leadership, communication, situation awareness and managing stress has increased significantly, and physicians have been assessed significantly more frequently for all NTS except managing stress and coping with fatigue (Table 3, Fig. 2).

Table 3 Physicians with simulation-based training in and assessment of non-technical skills
Fig. 2
figure 2

The changes in (a) simulation-based training in and (b) assessment of the generic non-technical skills within each professional group from 2011 to 2015. The ratios represent the relative frequencies (%) of 2015 divided by the relative frequencies (%) of 2011 across all four ordinal categories, with a ratio = 1 (dashed line) indicating no change in relative frequency and a ratio < 1 or > 1 respectively a decrease or an increase in frequency. Missing bars are due to categories with no data in one or both of the years surveyed, and thus, no computable ratio

In 2011, pilots reported to be assessed more frequently than physicians, while no significant difference was found regarding simulation-based training [8]. The bar plots indicate a further increase in the frequency of training and assessments for the pilots, but these changes were not significant with the exception of training and assessment of “situation awareness” and “managing stress” (Table 4, Fig. 2).

Table 4 Pilots with simulation-based training in and assessment of non-technical skills

HCMs appeared to be the professional group with the highest frequency of training and assessment in 2011, although not significantly different from the pilots [8]. We found a further and significant increase in the frequency of HCMs of simulation-based training in decision-making, communication, teamwork and managing stress. No significant changes were noted for assessment of any of the NTS categories. (Table 5, Fig. 2).

Table 5 HEMS crew members (HCM) with simulation-based training in and assessment of non-technical skills

Training and assessment based on employer

The crew members can be separated with respect to employer. Of the respondents, 49% were employed by the flight operator (HCMs and pilots) and 51% were working for the health enterprise (physicians) compared to 47 and 53%, respectively in the previous survey [8].

In 2011, health enterprise employees experienced significantly less frequent training and assessment than flight operator personnel for all NTS categories [8]. In our study, flight operator employees were reporting a significant increase in the frequency of both training and assessment of all NTS except “leadership” and “coping with fatigue” (Table 6). Even though the physicians were the group with most categories with significant increase in training and assessment in the period (Table 3), the significant differences based on employment status still exist for all categories except “leadership” (Table 6).

Table 6 Flight operator employees and health enterprise employees with simulation-based training in and assessment of non-technical skills


Training of non-technical skills

To deliver high quality of care and patient safety, training in technical skills is important to be competent in critical care procedures [14]. Non-technical skills are essential to complement the technical skills in a work setting such as HEMS. Deficiencies in communication and teamwork are frequent contributors to adverse events in health care [15]. There is also increasing awareness about the positive influence of teamwork on clinical performance [16, 17] and clinical outcomes [18, 19].

Even though the theoretical basis and the evidence regarding educational methods to enhance patient safety using NTS training are still limited [10], both simulation and classroom-based training has been found to improve teamwork processes [15]. An interdisciplinary team training program using in-situ simulation gave a statistically significant and persistent improvement in perinatal morbidity [20]. Similar results have been found in surgical outcome after team training of operating room personnel [19]. Simulation-based team training seems to be the most prominent mode of training in the literature [15].

Duration and frequency of training varies, and there is currently limited, but emerging, evidence that provides insight into the frequency of retraining needed to maintain effective teamwork skills [15]. Significant improvement has been found for critical care providers at 6 and 12 months post-training [21], and studies on simulation based training in neonatal resuscitation seems to favour low dose, high frequency training [22]. This points in the direction of at least annual training, similar to common practice for crew resource management (CRM) training in aviation.

The content and schedule of training in technical skills need to be tailored due to variations in mission profiles and exposure to different procedures [14]. Human errors, on the other hand, are not limited to inexperienced clinicians, and NTS training is therefore equally important to all. So far, a consensus regarding the content of team training has not been achieved, but the most commonly targeted teamwork competencies are communication, situational awareness and leadership [15]. In addition to these, decision-making, teamwork, managing stress and coping with fatigue are often included in non-technical skills evaluation schemes.

Assessment versus training

Assessment is the process of observing, recording, interpreting and evaluating individual performance and serves different purposes: to audit the level of skills of individuals or units, but also to evaluate training programs [11]. A number of non-technical skills rating frameworks, behavioural marker systems, have been developed for health-care domains closely related to the air ambulance setting [23,24,25,26], but a tool for assessment of non-technical skills for HEMS such as the AeroNOTS, has just recently been developed and yet not fully validated [27]. Generally, the frequency of assessment was lower than the frequency of simulation-based training for all three professional groups in our study. This result underlines the undone work in using assessment tools systematically in HEMS.

Training in Norwegian HEMS

Norwegian HEMS providers have a contractual mandatory training program in rescue and flight operative procedures, including recurrent flight simulator training for pilots and HCMs. Medical training, simulation-based or otherwise, depend on local initiative and commitment. In the study of Abrahamsen, physicians underwent significantly less frequent simulation-based training compared to the other groups [8]. In our study, physicians were the one group with a significant increase in most NTS categories, and thus, an important contributor to the overall increase in the frequency of training in the Norwegian HEMS. The before-mentioned initiatives with in-situ simulation [13] and the all crew training camp [12] may be one explanation to this result. The proportion of physicians training currently seems to be at the level of the other groups in 2011, but they still train significantly less than flight operative employees. Thus, a great potential for simulation-based training still exists among the HEMS physicians.

Coping with fatigue

The results from the different professional groups were inconsistent regarding each of the generic NTS, and with the limitation in response rate and sample size in our survey, these results should not be over-interpreted. For coping with fatigue, on the other hand, we did not find significant increase for any professional group, despite the finding from 2011 where all professional groups received limited training. This may be seen as a paradox since the non-technical performance of critical care air transfer clinicians is impaired when they are fatigued [28], and fatigue training seems to improve safety and health outcome for EMS personnel [29]. Fatigue and stress management are usually included in training programs, although it has been questioned whether it is appropriate to include these topics in assessment schemes of NTS. Both can be difficult to detect and rate unless extreme symptoms are displayed, in which other skills will be affected [11]. Another influencing factor may be the lack of a consensus on the definition of fatigue and a standardized survey instrument to measure fatigue among EMS worker groups. Only a limited number of tools used in other settings for assessment of fatigue exist, and research focused on development and testing of fatigue survey instruments tailored specifically for emergency medical services is needed [30]. The on-going research project in Norwegian HEMS in fatigue risk management will hopefully contribute to developing useful tools for fatigue training and assessment.


Our study was part of a combined survey of both ground and air ambulance with more than 5000 invited participants, and thus, the same follow up with personal reminders to all invited as the survey of Abrahamsen [8], was not feasible. Our response rate is therefore noticeably lower, but the distribution in professional groups, prehospital experience and geographical location was largely similar (Table 1). We do not know, however, if personnel who have undergone training were more likely to respond to our survey or not, which could result in a non-responder bias and possibly more significant changes than otherwise. The results should be interpreted according to these limitations with an emphasis on the major lines and not detailed results.

In both surveys, respondents were asked to report exclusively on the frequency of interdisciplinary prehospital simulation training. We cannot, nevertheless, exclude that pilots and HCMs may have reported on mandatory flight operative training and that this may explain the better results for these groups in both surveys. We also cannot exclude that physicians may have reported on intra-hospital training.

When asked retrospective to specify the number of training sessions and assessments, some uncertainty must be expected. We have mainly based our conclusions on the dichotomized data, “no training” or “some training”, which we have assumed more reliable. Ideally, a longer period between the two surveys would be preferable. This was not possible as our study was a part of a larger research project.

Finally, as discussed earlier, in order to fully understand the effect of simulation training on patient outcome, further research is needed.


The frequency of simulation-based training and assessment of NTS has increased significantly in Norwegian HEMS. Physicians seem to be adjusting to the training culture of other professional groups in HEMS, but still, there is a great potential for improving training frequency and volume among the HEMS physicians. Systematic assessment of NTS, including fatigue management, should be a future focus area in HEMS.



Aeromedical non-technical skills


HEMS Crew Member


Helicopter Emergency Medical Service


Non-technical skills


Search and rescue


  1. Parmentier-Decrucq E, Poissy J, Favory R, Nseir S, Onimus T, Guerry M-J, Durocher A, Mathieu D. Adverse events during intrahospital transport of critically ill patients: incidence and risk factors. Ann Intensive Care. 2013;3(1):10

    Article  Google Scholar 

  2. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2009;53:143–51

    Article  CAS  Google Scholar 

  3. Künzle B, Kolbe M, Grote G. Ensuring patient safety through effective leadership behaviour: a literature review. Safety Sci. 2010;48:1–17

    Article  Google Scholar 

  4. Rosenman ED, Branzetti JB, Fernandez R. Assessing team leadership in emergency medicine: the milestones and beyond. J Grad Med Educ. 2016;8(3):332–40

    Article  Google Scholar 

  5. MacDonald RD, Banks BA, Morrison M. Epidemiology of adverse events in air medical transport. Acad Emerg Med. 2008;15:923–31

    Article  Google Scholar 

  6. Dalto JD, Weir C, Thomas F. Analyzing communication errors in an air medical transport service. Air Med J. 2013;32(3):129–37

    Article  Google Scholar 

  7. Bigham BL, Buick JE, Brooks SC, Morrison M, Shojania KG, Morrison LJ. Patient safety in emergency medical services: a systematic review of the literature. Prehospital Emerg Care. 2012;16:20–35

    Article  Google Scholar 

  8. Abrahamsen HB, Sollid SJM, Öhlund LS, Røislien J, Bondevik GT. Simulation-based training and assessment of non-technical skills in the Norwegian helicopter emergency medical services: a cross-sectional survey. Emerg Med J. 2015;32:647–53

    Article  Google Scholar 

  9. Winkelmann M, Friedrich L, Schroter C, Flemming A, Eismann H, Sieg L, et al. Simulator-based air medical training program Christoph life: from concept to course. Air Med J. 2016;35(4):242–6

    Article  Google Scholar 

  10. Gordon M, Darbyshire D, Baker P. Non-technical skills training to enhance patient safety: a systematic review. Med Educ. 2012;46(11):1042–54

    Article  Google Scholar 

  11. Flin R, O’Connor P, Crichton M. Safety at the sharp end. A guide to non-technical skills. Aldershot: Ashgate; 2008.

    Google Scholar 

  12. Martinsen J. Observation and rating HEMS crew in non-technical skills, CRM medical simulation in Norwegian air ambulance. Scand J Trauma Resusc Emerg Med. 2015;23(2):A21

    Article  Google Scholar 

  13. Bredmose P, Sollid SJM. Weekly simulation for an on call helicopter emergency medical crew: feasible or impossible? Scand J Trauma Resusc Emerg Med. 2015;23(2):A23

    Article  Google Scholar 

  14. Sollid SJM, Bredmose P, Nakstad AR, Sandberg M. A prospective survey of critical care procedures performed by physicians in helicopter emergency medical service: is clinical exposure enough to stay proficient? Scand J Trauma Resusc Emerg Med. 2015;23(45)

  15. Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf. 2014;23(5):359–72

    Article  Google Scholar 

  16. Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. Br J Anaesth. 2013;110(4):529–44

    Article  CAS  Google Scholar 

  17. Gjeraa K, Moller TP, Østergaard D. Efficacy of simulation-based trauma team training of non-technical skills. A systematic review. Acta Anaesthesiol Scand. 2014;58(7):775–87

    Article  CAS  Google Scholar 

  18. Salas E, Rosen MA. Building high reliability teams: progress and some reflections on teamwork training. BMJ Qual Saf. 2013;22(5):369–73

    Article  Google Scholar 

  19. Neily J, Mills PD, Young-Xu Y, Carney BT, West P, Berger DH, et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304(15):1693–700

    Article  CAS  Google Scholar 

  20. Riley W, Davis S, Miller K, Hansen H, Sainfort F, Sweet R. Didactic and simulation nontechnical skills team training to improve perinatal patient outcomes in a community hospital. Jt Comm J Qual Patient Saf. 2011;37(8):357–64

    Article  Google Scholar 

  21. Mayer CM, Cluff L, Lin W-T, Willis TS, Stafford RE, Williams C, et al. Evaluating efforts to optimize TeamSTEPPS implementation in surgical and pediatric intensive care units. Jt Comm J Qual Patient Saf. 2011;37(8):365–74

    Article  Google Scholar 

  22. Mduma E, Ersdal H, Svensen E, Kidanto H, Auestad B, Perlman J. Frequent brief on-site simulation training and reduction in 24-h neonatal mortality—an educational intervention study. Resuscitation. 2015;93:1–7

    Article  Google Scholar 

  23. Flin R, Patey R, Glavin R, Maran N. Anaesthetists’ non-technical skills. Br J Anaesth. 2010;105(1):38–44

    Article  CAS  Google Scholar 

  24. Flin R, Maran N. Identifying and training non-technical skills for teams in acute medicine. Quality & safety in health care. 2004;13:i80–4

    Article  Google Scholar 

  25. Ummenhofer W, Amsler F, Sutter PM, Martina B, Martin J, Scheidegger D. Team performance in the emergency room: assessment of inter-disciplinary attitudes. Resuscitation. 2001;49(1):39–46

    Article  CAS  Google Scholar 

  26. Robertson ER, Hadi M, Morgan LJ, et al. Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. Roma PG, ed PLoS ONE. 2014;9(3):e90320

    Article  Google Scholar 

  27. Myers JA, Powell DMC, Psirides A, Hathaway K, Aldington S, Haney MF. Non-technical skills evaluation in the critical care air ambulance environment: introduction of an adapted rating instrument--an observational study. Scand J Trauma Resusc Emerg Med. 2016;24:24

    Article  Google Scholar 

  28. Myers JA, Powell DMC, Aldington S, Sim D, Psirides A, Hathaway K, et al. The impact of fatigue on the non-technical skills performance of critical care air ambulance clinicians. Acta Anaesthesiol Scand. 2017;61(10):1305–13

    Article  CAS  Google Scholar 

  29. Barger LK, Runyon MS, Renn ML, Moore CG, Weiss PM, Condle JP, et al. Effect of fatigue training on safety, fatigue, and sleep in emergency medical services personnel and other shift workers: a systematic review and meta-analysis. Prehosp Emerg Care. 2018;22(sup1):58–68

    Article  Google Scholar 

  30. Patterson PD, Weaver MD, Fabio A, Teasley EM, Renn ML, Curtis BR, et al. Reliability and validity of survey instruments to measure work-related fatigue in the emergency medical services setting: a systematic review. Prehosp Emerg Care. 2018;22(sup1):17–27

    Article  Google Scholar 

Download references


The authors would like to thank Lufttransport RW AS, Norsk Luftambulanse AS and the medical leaders for their help in distributing and promoting the survey.


This study received no specific funding. KR received research grants from Norwegian Air Ambulance Foundation and Møre and Romsdal Hospital Trust and HL received funding from the Norwegian Ministry of Education and Research. The funders did not take part in any part of the study or manuscript.

Availability of data and materials

The datasets supporting the conclusions of this article are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations



KR, HBA, EBA, SJMS: Study design; LIKS, KR, HBA, SJMS, GTB: data collection; KR, HL: data analysis; KR: draft of the manuscript; all authors revised the manuscript and approved the final version.

Corresponding author

Correspondence to Kristen Rasmussen.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Norwegian Centre for Research Data (Ref. no. 2016/45723) and was exempted from ethical approval by the Regional Committee for Medical and Health Research Western Norway (Ref. no. 2015/2249). Consent to participate in the study was given at the start of the survey.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Questionnaire (English translation). (PDF 180 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rasmussen, K., Langdalen, H., Sollid, S.J.M. et al. Training and assessment of non-technical skills in Norwegian helicopter emergency services: a cross-sectional and longitudinal study. Scand J Trauma Resusc Emerg Med 27, 1 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: