Skip to main content

Residents working with Médecins Sans Frontières: training and pilot evaluation

Abstract

Background

Well-prepared humanitarian workers are now more necessary than ever. Essential to the preparation process are: clearly defined learning objectives, curricula tailored to the nuances of humanitarian settings, simulation-based training, and evaluation. This manuscript describes a training program designed to prepare medical residents for their first field deployment with Médecins Sans Frontières and presents the results of a pilot assessment of its effectiveness.

Methods

The training was jointly developed by the Research Center in Emergency and Disaster Medicine- CRIMEDIM of the Università del Piemonte Orientale, Novara, Italy, and the humanitarian aid organization Médecins Sans Frontières- Italy (MSF-Italy); the following topics were covered: disaster medicine, public health, safety and security, infectious diseases, psychological support, communication, humanitarian law, leadership, and job-specific skills. It used a blended-learning approach consisting of a 3-month distance learning module; 1-week instructor-led coaching; and a field placement with MSF. We assessed its effectiveness using the first three levels of Kirkpatrick’s training evaluation model.

Results

Eight residents took part in the evaluation. Four were residents in emergency medicine, 3 in anesthesia, and 1 in pediatrics; 3 of them were female and the median age was 31 years. Two residents were deployed in Pakistan, 1 in Afghanistan, 1 in the Democratic Republic of Congo, 1 in Iraq, 2 in Haiti and 1 on board of the MSF Mediterranean search & rescue ship. Mean deployment time was 3 months. The average median score for the overall course was 5 (excellent). There was a significant improvement in post-test multiple choice scores (p = 0.001) and in residents’ overall performance scores (P = 0.000001).

Conclusion

Residents were highly satisfied with the training program and their knowledge and skills improved as a result of participation.

Trial registration

This study was approved by the Institutional Ethics Committee (date 24-02-2016, study code UPO.2015.4.10).

Introduction

Widespread global health inequalities and the resulting shortage of humanitarian health workers have led to an increased presence of young doctors in disaster and humanitarian crises [1]. However, although well-prepared humanitarian workers are more necessary than ever [2], traditional medical education struggles to meet the demands posed by globalization and the dramatic escalation of violence [3].

Training objectives for physicians working with Médecins Sans Frontières (MSF) differ from the set of skills acquired in medical schools and residency programs. These professionals are in fact confronted with unique challenges and ethical dilemmas [3] including, but not limited to, different spectra of diseases, limited resources, cultural diversity, and social disruption. For these reasons, training objectives should be set far beyond individual job-specific skills and incorporate a set of core medical and non-medical competencies that all humanitarian workers must possess [4]. Essential to the preparation process before first deployment are therefore clearly defined learning objectives, curricula tailored to the nuances of humanitarian settings, state-of-the-art teaching tools, including high fidelity simulation, and assessment to determine competency [5].

Despite the need for curricula built on a testable package of knowledge and skills, evaluating the effectiveness of training programs is generally not an essential component of preparedness for humanitarian health workers [6], or at best is limited to measuring satisfaction or knowledge. Assessing whether students improve their ability to handle complex situations is of paramount importance to guarantee the best outcomes for vulnerable populations; for this reason, this manuscript describes a training program designed to prepare medical residents for their first deployment with Médecins Sans Frontières and presents the results of a pilot assessment of its effectiveness.

Materials and methods

Program development

Target Population Survey

In 2012, we conducted a nationwide survey to investigate the interest of young doctors in humanitarian assistance [7]. The survey included all residents in anesthesia and intensive care in Italy. Out of 924 respondents (RR 67.8%), 691 (74.7%) would have liked to make a major contribution to international humanitarian health care during their residencies and 897 (97%) would welcome specific training prior to deployment. Building on these results, we assumed that our training program would be of interest to doctors training in this discipline.

Institutions involved

In 2013, the Research Center in Emergency and Disaster Medicine-CRIMEDIM of the Università del Piemonte Orientale (Novara, Italy) in collaboration with Médecins Sans Frontières Italy (MSF-Italy) developed Humanitarian Medic [8], a competency-based training program delivered annually to senior residents in anesthesia and critical care before their first deployment to MSF field missions. In 2015, the course was expanded to residents in emergency medicine and pediatrics.

Candidate selection

The number of participants cannot not exceed 10 per iteration; the course includes national and international senior residents (IV-V year) in anesthesia & critical care, emergency medicine, and pediatrics. Selection criteria coincide with the minimum MSF standard requirements for humanitarian workers, namely:

  • At least B1 level proficiency in French and English (the languages of health care in the United Nations) according to the European Language Framework;

  • Willingness to participate in international humanitarian field projects, including in armed conflict areas or following natural or man-made disasters;

  • Flexibility and a positive attitude to working in multicultural contexts.

Prior participation in international cooperation projects or humanitarian emergency response programs is considered an asset but is not mandatory for application.

Candidate selection is carried out in a biphasic fashion by a recruitment commission composed of two CRIMEDIM investigators and two recruiters from MSF-Italy human resources department. Participants are first screened based on their curriculum vitae, self-assessed theoretical and practical skills and the results obtained in an on-line French and English language test. Subsequently, the best candidates are selected for interview.

Curriculum

Educational needs were established on the basis of an expert opinion survey [9], round tables with CRIMEDIM and MSF field experts, and a literature review of published competency sets for humanitarian workers [10]. Since our training program targeted health workers but was meant to be extendable in the future to other sectors operating under the umbrella of humanitarian aid, four manuscripts were selected on the basis of their cross-sectorial approach [6] and definition of discipline-specific competencies relevant to our audience [11,12,13]. These competency sets served as a foundational basis for the course curriculum and were translated into 10 cross-sectorial and 1 profession-specific competency domain for each specialty [Table 1]. Learning objectives were phrased according to Bloom’s Taxonomy. For each objective, a series of measurable performance goals was further developed. Curricula, learning and performance objectives were reviewed and validated by consensus between CRIMEDIM and MSF-Italy working groups.

Table 1 Competency domains, general learning objectives and some performance objectives at the basis of the training curriculum

Delivery method

The course intends to expose participants to a blended-learning experience consisting of 3months of distance, self-directed learning and 1 week of residential instructor-led teaching. E-learning takes place through the Modular Object-Oriented Dynamic Learning Environment (MOODLE) educational software hosted on CRIMEDIM’s servers. The platform works as a content-driven learning model, hosting 11 e-modules and video lectures and offering a suite of tools and online-multiplayer-virtual exercises. The residential phase takes place at the SIMNOVA simulation center in Novara, Italy, and includes class-room sessions, table-top exercises, and group discussions with emphasis on high fidelity and outdoor real-size simulation exercises. Scenarios are designed based on the equipment, drugs and diagnostic tools available in MSF field projects and residents are exposed to the challenges most commonly encountered in daily activities. In clinical management scenarios, actors comply with the dress code of the country where the scenario is based and are also trained to act as typical members of the local staff.

E-learning materials and best performances for simulation exercises are jointly developed based on current international guidelines and the typical resources available in real-life MSF missions. Upon successful completion of both phases, students receive a certificate of completion and are deployed to MSF field projects as local staff supervisors to work as part of the hospital duty roster.

Evaluation

Participants

The first two editions in 2013 and 2014 were not included in the evaluation but were offered on a pilot basis to test the feasibility of the project from an organizational standpoint and refine the course contents according to the feedback provided by students and MSF field supervisors.; only participants to the 2015 course iteration (n = 8) were included in the evaluation. All were Italian, four were residents in emergency medicine, 3 in anesthesia, and 1 in pediatrics; 3 of them were female and median age was 31 years old. Two residents were deployed in Pakistan, 1 in Afghanistan, 1 in the Democratic Republic of Congo, 1 in Iraq, 2 in Haiti and 1 on board of the MSF Mediterranean search & rescue ship. Mean deployment time was 3 months.

Method

The Kirkpatrick’s evaluation model has recently been used to evaluate training programs for health providers and focuses on the sequential assessment of the following levels [14]:

  • Level 1- Reaction: measures students’ satisfaction with the program;

  • Level 2- Learning: measures improvement in knowledge, attitudes and skills;

  • Level 3- Behavior: measures the transfer of learning to the workplace;

  • Level 4- Results: measures the objective changes occurred as a result of participation in the training program.

To determine the effectiveness of our course, we tested levels 1 to 3 using a prospective, observational, single-cohort study.

Evaluation plan and evaluation tools

We designed the evaluation plan according to the recommendations of Kirkpatrick et al. [15] [Fig. 1]:

  • We assessed Reaction using a 5-point Likert scale questionnaire with a separate space for commentaries and personal opinions.

  • We evaluated the three dimensions of Learning separately in a pre- and post-test [Fig. 1] as follows:

    • Knowledge with a 30-question-multiple-choice test.

    • Attitude with a 12-question-5-point Likert scale questionnaire. In this study, the term “attitude” was defined as the students’ positive or negative predisposition toward the competency domains at the basis of the course.

    • Skills with simulation-based performance tests, in which each student acted as lead physician in the management of a critically-ill patient in a low-resource emergency room. To decrease the potential impact that reiterative exposure to simulation exercises during the residential phase might have had on post-test performance, students attended standardized simulation tutorials and managed a number of simulated cases before entering the pre-test scenario. Pre- and post-test scenarios progressed on a predefined fashion according to the cue system described by Kim et al. [16] We videotaped resident’s performance and passed the material on to an external independent evaluator, who rated all students according to the validated Italian translation (Translated Italian Global Rating Scale- TIGR) [17] of the Ottawa Global Rating Scale (Ottawa GRS) [16]. This scale uses a 7-point semi-anchored design to evaluate residents’ overall performance and their leadership, problem solving, situational awareness, resource utilization, and communication skills.

    • Pre- and post-test were different tests with the same level of difficulty.

  • Every resident was followed during the mission by his immediate MSF field supervisor who qualitatively assessed improvements in behavior using the MSF standard evaluation form for first missioners (form available upon request).

  • MSF field supervisors were blinded to the students’ completion of the training program before deployment. This was necessary to prevent students’ participation from influencing the judgement of supervisors and ensure that evaluations were carried out as usual for any MSF first missioner.

  • All the evaluation tests/forms and case scenarios used to conduct the evaluation are available upon request.

Fig. 1
figure1

Graphic depiction of the evaluation process. Overall, residents participated in a face- to-face course opening session where Level 2-pre-test was administered. Level 1 and Level 2 (post-test) were assessed at the end of the residential course. Behavior was evaluated at the end of students’ field missions

Statistical tests and data analysis

The statistical analysis of Level 2 (Learning) was conducted by taking the difference in pre- and post-test scores as measured with a multiple-choice test as the primary objective. The difference in pre-and post-overall performance was defined as the secondary objective. We defined null hypotheses as no difference between the before and after multiple-choice test scores and no difference between the before and after overall performance. Alternative hypotheses were that both multiple choice test scores and overall performance scores would change significantly after completion of the course.

We tested the null hypothesis of the primary objective of no difference between pre- and post-test scores against the two-sided alternative hypothesis of significant difference. The null hypothesis of the secondary objective of no difference between pre- and post-training overall performance scores was tested against the two-sided alternative of significant difference. Statistical analysis was performed using “R: A language and environment for statistical computing.” (R Development Core Team, Vienna, Austria). Null and alternative hypotheses and the statistical methods were completely specified before data collection. Differences between groups for the primary and secondary objectives were assessed using paired t-tests. P-values of less than .05 were considered significant for all tests. Two-sided alternative hypotheses were used in all cases.

Ethical clearance

To ensure anonymity and confidentiality during the entire evaluation process, we assigned a tracking number to each participant. This number was then reported on answer sheets, evaluation forms and videotape labels. All students signed the informed consent. This study was approved by the institutional Ethics Committee (date 24-02-2016, study code UPO.2015.4.10).

Results

Reaction

All residents rated the course as “excellent” and strongly agreed with the statement “I would recommend the course to other doctors in training”. All participants emphasized their satisfaction with both course contents and mode of delivery. In particular, high-fidelity simulation exercises were highly appreciated. Two students expressed the view that the course schedule was too tight and one suggested adding more training in negotiations techniques.

Learning

There was a significant improvement in post-test multiple-choice scores when compared to pre-test scores (p = .001) (mean effect: 10.4/30; 95% CI: 5.7 to 15.0) [Fig. 2] and also a significant improvement in residents’ overall performance scores (P = .000001) (mean effect: 3; 95% CI: 2.4 to 3.6) [Fig. 3]. The median score for all other fields also improved after the training course (median differences: Leadership 3.3; Problem solving 2.6; Situational Awareness 3; Resource Utilization 3.5; Communication 2.7). No differences were detected in attitudes scores before and after the course.

Fig. 2
figure2

Boxplot representing multiple choice test scores before and after the course

Fig. 3
figure3

Boxplot representing (TIGR) overall performance test scores before and after the course

Behavior

For most participants, MFPs highlighted the following strengths: compliance with MSF standards and principles (7), flexibility (6), good team working skills (6) and cross-cultural sensitivity (3). All residents (8) were recommended for future MSF missions [Table 2].

Table 2 Summary of evaluations from field supervisors for each candidate

Discussion

Expatriate health providers have been observed to be ill-prepared during recent humanitarian emergencies due to lack of experience in international relief and inadequate understanding of the local context [4, 18]. For this reason, the international humanitarian community has been drawing attention to the compelling need for competency-based training curricula based on a standard set of cross-cutting and profession-specific competencies [4, 19]. Since young doctors, born in a new era of highly-specialized medicine, have been increasing their presence in international aid projects, good preparation and performance oversight are paramount to guarantee best practice also in resource-strained settings [9].

To our knowledge, this is the first study that describes the implementation and evaluation of a course based on published cross-sectorial and profession-specific competencies, jointly developed by an academic center and a robust humanitarian organization. Interestingly, while 61% of training programs for humanitarian workers in Europe [19] are defined as “competency-based”, none of them incorporates previously published competency sets. Evaluating the effectiveness of training programs is necessary to ensure credibility and decide whether they should be continued or not. In this regard, the assessment of a course for humanitarian workers using the first three levels of the Kirkpatrick’s evaluation model and including high fidelity simulation represents a real novelty.

Overall, students’ satisfaction (Level 1) with the program was high. In particular, the delivery method and the residential phase were highly appreciated. It is worth noting that, aside from course curriculum and students’ previous academic background, the structure of the program and the educational environment also play a central role in learning [20]. Over the last years, medical education has shifted toward different delivery modes in an attempt to achieve better educational outcomes. The combination of face-to-face lectures and online teaching, defined as blended learning, strengthens the interaction between course participants, lecturers and resources [21] and represents a flexible pedagogical system [22].

Our study showed that students’ competency in simulated humanitarian scenarios increased after course completion (Level 2). The term “competency” is defined as the set of knowledge, attitudes and skills required to accomplish a task. A competency-based training must ensure the acquisition of a theoretical understanding of relevant concepts in the field of humanitarian assistance (e.g., learning the Simple Triage And Rapid Treatment (START) triage algorithm), promote a positive predisposition toward the working methods and actions to be adopted in low-resource settings (e.g., recognizing the importance of applying the START triage), and develop students’ practical skills (e.g., conducting good quality triage in case of a mass casualty event). Yoon et al. [23] used the Kirkpatrick model to evaluate a continuing professional development training for physicians and physician assistants. In their study, a single 5-point Likert scale form filled by trainers and trainees was used to assess learning. However, it is important to highlight that the inclusion of separate tests for knowledge (e.g., multiple choice), attitudes (e.g., Likert scale questionnaire), and skills (e.g., performance test) should be preferred whenever possible [14].

Our results reflect a clear improvement in participants’ knowledge and overall performance in a high-fidelity scenario, while no change in attitude was evident. A plausible explanation may lie in the selection process itself. Since all students were highly motivated and had decided to take part in the course on a voluntary basis, a positive attitude was to be expected.

In a recent study, Schwartz et al. highlighted the prominent role that simulations, and particularly high-fidelity simulations, may play in enhancing residents’ skills in the management of complex cases [24]. Simulated environments are an invaluable setting to teach crisis resource management (CRM) skills [25], which are extremely important in humanitarian contexts. Some challenges commonly encountered in the field (e.g., communication barriers and shortage of resources) can be easily reproduced through simulation, giving students the chance to become acquainted with similar situations, receive feedback and improve their performance with no risks to patients. At the same time, simulated scenarios allow for the evaluation of performance objectives, reflecting how students would use in the field the competencies acquired through training [11]. In their review entitled “Transfer of learning and patient outcome in simulated crisis resource management”, Boet et al. found that CRM simulations improve not only learners behavior in the workplace but also, and more importantly, patient outcomes [26].

According to Kirkpatrick et al., [14] a positive reaction and evidence of improvement in learning do not necessarily lead to desired changes in behavior. The transfer of learning to the workplace is heavily conditioned by the so-called ‘work climates’, and these are clearly established by a supervisor’s reaction to students’ practical application of the competencies acquired. To promote an encouraging work environment, heads of department and supervisors should be informed about the students’ participation in the training program, and preferably be involved in its development [14]. In our case, keeping MSF field evaluators blinded was mandatory to prevent biases; however, all participants received very good feedbacks. The reason for this may be the fact that educational needs were decided and endorsed by a panel of experts that included MSF training staff. This ensured that the practical concepts, organizational principles and techniques taught complied with the organization’s best practice standards.

The collaboration between an academic center and a robust humanitarian organization allowed us to demonstrate the effectiveness of a pre-deployment training course in improving participants’ learning. This may have several promising implications:

  1. 1.

    If properly trained, medical residents with no previous experience in the field could be deployed without compromising the quality of care delivered.

  2. 2.

    In countries where residents are authorized by contract to practice abroad for a certain period of time while maintaining their financial remuneration, non-governmental organizations (NGOs) could fill field gaps more rapidly by deploying well-prepared but inexpensive personnel.

  3. 3.

    From an organizational standpoint, agreements between NGOs and training centers would allow humanitarian staff to benefit from simulation-based training, which is presently the best approximation to real work in emergency and disaster contexts [27].

It is our hope that this collaborative initiative will serve worldwide as a model to bridge the gap between academia and field operations and contribute to the growth and professionalization of the humanitarian health sector.

Limitations

Despite our efforts to conduct this study thoroughly, a number of limitations should be considered.

The competency sets and skills at the basis of our training curriculum, albeit published and peer-reviewed, were never validated. However, the authors believe that basing the learning objectives of the course on the needs emerged from discussions with different groups of experts would go some way toward remedying this deficiency. Incorporating the input of trainers working for the NGO partner would also be a fair compromise in the absence of a globally recognized competency set for humanitarian workers.

The study dataset was limited to 2015 and the sample size used to test the effectiveness of this course was limited to 8 participants. Since our target population was composed of doctors still at an early stage of their careers, the tight enrollment criteria severely limited the number of eligible applicants for each iteration. Also, the evaluation process was logistically challenging and very resource consuming. All participants had to travel to Novara on purpose for the pre-test and their travel and living expenses had to be covered for 2 days both pre- and post-test. This prevented the evaluation to be repeated in the following editions, which would have positively impacted the sample size. Even though both primary and secondary outcomes improved significantly after the course, a larger study would be useful to confirm the significance of the specific changes found in each field of the TIGR evaluation scale.

This study only assessed the first three levels of the Kirkpatrick model; additionally, for level 3 (Behavior) no pre-test was conducted. Since the course was designed to prepare residents to their first deployment with MSF, exposition to the real workplace environment was only possible upon completion. All participants obtained very good assessments from field supervisors, which may suggest that the training program played a role in the quality of their respective performances in the field.

Level 4 (Results) measures the effect of students’ actions, which considering our target population should have translated into measurement of patients’ outcomes (e.g., decreased mortality). Taking into account the high turnover of doctors in humanitarian contexts and the diversity in the pattern of disease and affluence of patients depending on country, season, and ongoing environmental conditions (armed conflict, natural disaster, etc.), the influence on patients’ outcomes would have been very hard to ascertain.

Finally, this study did not include a control group. According to MSF policy, deploying untrained doctors in the field at this early stage of their careers would have gone against basic principles of best practice.

Conclusions

Over the last decade, the humanitarian community has stressed the need to improve the quality of response through further investments in training for aid workers. Residents were highly satisfied with our training program and their knowledge and skills in simulated humanitarian environments improved as a result of participation. The implementation of this project shows how academia can successfully partner with humanitarian aid organizations to promote the professionalization of future humanitarian health workers. Further studies should be conducted to assess whether training programs effectively increase the competence of humanitarian workers in the field and if this translates into improvement of patients’ outcomes or further advantages for deploying organizations.

Availability of data and materials

Complete data and evaluation forms/tests used are available upon request.

Abbreviations

MSF:

Médecins Sans Frontières

CRIMEDIM:

Research Center in Emergency and Disaster Medicine of the Università del Piemonte Orientale, Novara, Italy

MOODLE:

Modular Object-Oriented Dynamic Learning Environment

TIGR:

Translated Italian Global Rating Scale

Ottawa GRS:

Ottawa Global Rating Scale

START:

Simple Triage And Rapid Treatment

CRM:

Crisis resource management

NGOs:

Non-governmental organizations

References

  1. 1.

    Fouad FM, Sparrow A, Tarakji A, Alameddine M, El-Jardali F, Coutts AP, et al. Health workers and the weaponisation of health care in Syria: a preliminary inquiry for The Lancet–American University of Beirut Commission on Syria. Lancet. 2017;390:2516–26 Lancet Publishing Group.

    Article  Google Scholar 

  2. 2.

    Redmond AD, Mardel S, Taithe B, Calvot T, Gosney J, Duttine A, et al. A qualitative and quantitative study of the surgical and rehabilitation response to the earthquake in Haiti, January 2010. Prehosp Disaster Med. 2011;26:449–56.

    Article  Google Scholar 

  3. 3.

    Wass V, Southgate L. Doctors without Borders. Acad Med. 2017;92:441–3.

    Article  Google Scholar 

  4. 4.

    Burkle FM, Walls AE, Heck JP, Sorensen BS, Cranmer HH, Johnson K, et al. Academic affiliated training centers in humanitarian health, part I: program characteristics and professionalization preferences of centers in North America. Prehosp Disaster Med. 2013;28:155–62.

    Article  Google Scholar 

  5. 5.

    Cranmer H, Chan JL, Kayden S, Musani A, Gasquet PE, Walker P, et al. Development of an evaluation framework suitable for assessing humanitarian workforce competencies during crisis simulation exercises. Prehosp Disaster Med. 2014;29:69–74.

    Article  Google Scholar 

  6. 6.

    Burkle FM Jr, James JJ. Cross-disciplinary competency and professionalization in disaster medicine and public health. In: Gursky E, Hreckovski B, editors. Handbook for Pandemic and Mass-Casualty Planning and Response 2012. Amsterdam: IOS Press; 2012. p. 72–83. LJM.

    Google Scholar 

  7. 7.

    Ripoll Gallardo A, Ingrassia PL, Ragazzoni L, Djalali A, Carenzo L, Burkle FM, et al. Professionalization of anesthesiologists and critical care specialists in humanitarian action: a nationwide poll among italian residents. Prehosp Disaster Med. 2015;30:16–21.

    Article  Google Scholar 

  8. 8.

    Course HM. Available at https://crimedim.uniupo.it/humanitarian-medic/. Last accessed January 2020.

  9. 9.

    Djalali A, Ingrassia PL, Della Corte F, Foletti M, Gallardo AR, Ragazzoni L, et al. Identifying deficiencies in national and foreign medical team responses through expert opinion surveys: implications for education and training--ERRATUM. Prehosp Disaster Med. 2015;30:224.

    Article  Google Scholar 

  10. 10.

    Ripoll Gallardo A, Djalali A, Foletti M, Ragazzoni L, Della Corte F, Lupescu O, et al. Core competencies in disaster management and humanitarian assistance: a systematic review. Disaster Med Public Heal Prep. 2015;9:430–9.

    Article  Google Scholar 

  11. 11.

    Schultz CH, Koenig KL, Whiteside M, Murray R. National Standardized all-Hazard Disaster Core Competencies Task F. development of national standardized all-hazard disaster core competencies for acute care physicians, nurses, and EMS professionals. Ann Emerg Med. 2012;59:196–208 e1.

    Article  Google Scholar 

  12. 12.

    Suchdev PS, Shah A, Derby KS, Hall L, Schubert C, Pak-Gorstein S, et al. A proposed model curriculum in global child health for pediatric residents. Acad Pediatr. 2012;12:229–37.

    Article  Google Scholar 

  13. 13.

    Rossler B, Marhofer P, Hupfl M, Peterhans B, Schebesta K. Preparedness of anesthesiologists working in humanitarian disasters. Disaster Med Public Heal Prep. 2013;7(4):408–12.

    Article  Google Scholar 

  14. 14.

    Kirkpatrick DL San Francisco, CA: Berret-Koehler Publishers; 2006 and KJDETPTFL 3rd ed.

  15. 15.

    Kirkpatrick DL, Kirkpatrick JD. Implementing the four levels: a practical guide for effective evaluation of training programs. San Francisco: San Francisco CB-KP; 2007.

  16. 16.

    Kim J, Cardinal P, Chiu M, Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa critical care medicine, high-Fidelity simulation, and crisis resource management I study. Crit Care Med. 2006;34:2167–74.

    Article  Google Scholar 

  17. 17.

    Franc JM, Verde M, Gallardo AR, Carenzo L, Ingrassia PL. An Italian version of the Ottawa crisis resource management global rating scale: a reliable and valid tool for assessment of simulation performance. Intern Emerg Med. 2017;12:651–6.

    Article  Google Scholar 

  18. 18.

    Van Hoving DJ, Wallis LA, Docrat F, De Vries S. Haiti disaster tourism--a medical shame. Prehosp Disaster Med. 2010;25(3):201–2 Available from: https://www.ncbi.nlm.nih.gov/pubmed/20586008.

    Article  Google Scholar 

  19. 19.

    Ingrassia PL, Foletti M, Djalali A, Scarone P, Ragazzoni L, Corte FD, et al. Education and training initiatives for crisis management in the European Union: a web-based analysis of available programs. Prehosp Disaster Med. 2014;29:115–26.

    Article  Google Scholar 

  20. 20.

    Hutchinson L. Educational environment. BMJ. 2003;326:810–2.

    Article  Google Scholar 

  21. 21.

    Garrison DRKH. Blended learning: uncovering its transformative potential in higher education. IHE. 2004;7(2):95–105.

    Google Scholar 

  22. 22.

    Lewin LO, Singh M, Bateman BL, Glover PB. Improving education in primary care: development of an online curriculum using the blended learning model. BMC Med Educ. 2009;9:33.

    Article  Google Scholar 

  23. 23.

    Yoon HB, Shin JS, Bouphavanh K, Kang YM. Evaluation of a continuing professional development training program for physicians and physician assistants in hospitals in Laos based on the Kirkpatrick model. J Educ Eval Heal Prof. 2016;13:21.

    Article  Google Scholar 

  24. 24.

    Schwartz KR, Prentiss KA. Simulation in pre-departure training for residents planning clinical work in a low-income country. West J Emerg Med. 2015;16:1166–72.

    Article  Google Scholar 

  25. 25.

    Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA. Simulation-based training in anesthesia crisis resource management (ACRM): a decade of experience. Simul Gaming. 2001;32:175–93.

    Article  Google Scholar 

  26. 26.

    Boet S, Bould MD, Fung L, Qosa H, Perrier L, Tavares W, et al. Transfer of learning and patient outcome in simulated crisis resource management: a systematic review. Can J Anaesth. 2014;6:571–82.

    Article  Google Scholar 

  27. 27.

    Amat Camacho N, Hughes A, Burkle FM Jr, Ingrassia PL, Ragazzoni L, Redmond A, et al. Education and training of emergency medical teams: recommendations for a global operational learning framework. PLoS Curr. 2016;8.

Download references

Acknowledgements

The authors thank all lecturers, assistants and actors involved in the training and simulations. They thank Pier Luigi Ingrassia, director of the simulation center SIMNOVA of the Università del Piemonte Orientale, for lending the simulation facilities and Katia Ansalone for her professional writing services.

Funding

None.

Author information

Affiliations

Authors

Contributions

ARG is anesthesia consultant at Maggiore Hospital, School of Medicine, in Novara and postdoctoral fellow at CRIMEDIM, Novara, Italy. She was first investigator and training program developer and contributed to the curriculum design, implementation of the evaluation and data collection and interpretation. She drafted the manuscript and gave final approval for the final version to be published; she is accountable for the accuracy and integrity of every part of the study. LR is postdoctoral fellow at CRIMEDIM, Novara, Italy. He contributed as training program developer and to curriculum design and training implementation. EM is pool manager at Médecins Sans Frontières-Italy, Rome, Italy. He contributed as training program developer and to curriculum design. He was also accountable for the matching of residents to their missions in the field. GM is anesthesia consultant at Maggiore Hospital, School of Medicine, Novara, Italy. She acted as main assistant and organizer for the SIMNOVA simulation exercises. JMF is clinical professor of Emergency Medicine at the University of Alberta, Edmonton, AB, Canada, and visiting professor of Disaster Medicine at the Università del Piemonte Orientale, Novara, Italy. He contributed to study design and data analysis. AC is an Anesthesia and Intensive Care Resident in and CRIMEDIM research assistant, Università del Piemonte Orientale, Novara, Italy. He contributed to the organization of the training program and its evaluation. FDC is full professor at the department of Anesthesia and Intensive Care, Maggiore Hospital, School of Medicine and director of CRIMEDIM, Novara, Italy. He contributed to the organization of the training program and its evaluation. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Alba Ripoll-Gallardo.

Ethics declarations

Ethics approval and consent to participate

All students signed the informed consent. This study was approved by the institutional Ethics Committee (date 24-02-2016, study code UPO.2015.4.10).

Consent for publication

All authors read the manuscript and agreed to its submission and publication in the Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ripoll-Gallardo, A., Ragazzoni, L., Mazzanti, E. et al. Residents working with Médecins Sans Frontières: training and pilot evaluation. Scand J Trauma Resusc Emerg Med 28, 86 (2020). https://doi.org/10.1186/s13049-020-00778-x

Download citation

Keywords

  • Humanitarian aid
  • Education
  • Residents
  • Evaluation
  • Low-resource environments
  • Simulation
  • Training
  • E-learning