Skip to content


  • Original research
  • Open Access

A tailored e-learning program to improve handover in the chain of emergency care: a pre-test post-test study

  • 1Email author,
  • 2,
  • 3,
  • 2,
  • 3,
  • 2,
  • 4, 5 and
  • 1, 6
Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine201523:33

  • Received: 27 August 2014
  • Accepted: 9 April 2015
  • Published:



To standardize patient handover in the chain of emergency care a handover guideline was developed. The main guideline recommendation is to use the DeMIST model (Demographics, Mechanism of Injury/illness, Injury/Illness, Signs, Treatment given) to structure pre-hospital notification and handover. To benefit from the new guideline, guideline adherence is necessary. As adherence to guidelines in emergency care settings is variable, there is a need to systematically implement the new guideline. For implementation of the guideline we developed a e-learning program tailored to influencing factors. The aim of the study was to evaluate the effectiveness of this e-learning program to improve emergency care professionals’ adherence to the handover guideline during pre-hospital notification and handover in the chain of emergency medical service (EMS), emergency medical dispatch (EMD), and emergency department (ED).


A prospective pre-test post-test study was conducted. The intervention was a tailored e-learning program that was offered to ambulance crew and emergency medical dispatchers (n=88). Data on adherence included pre-hospital notifications and handovers and were collected through observations and audiotapes before and after the e-learning program. Data were analyzed using X2-tests and t-tests.


In total, 78/88 (88.6%) professionals followed the e-learning program. During pre- and post-test, 146 and 169 handovers were observed respectively. After the e-learning program, no significant difference in the number of handovers with the DeMIST model (77.9% vs. 73.1%, p=.319) and the number of handovers with the correct sequence of the DeMIST model (69.9% vs. 70.5%, p=.159) existed. During the handover, the number of questions by ED staff and interruptions significantly increased from 49.0% to 68.9% and from 15.2% to 52.7% respectively (both p=.000). Most handovers were performed after patient transfer, this did not change after the intervention (p=.167). The number of handovers where information was documented during handover slightly increased from 26.9% to 29.3% (p=.632).


The tailored e-learning program did not improve adherence to a handover guideline in the chain of emergency care. Results show a relatively high baseline adherence rate to usage and correct sequence of the DeMIST model. Improvements in the handover process can be made on the documentation of information during handover, the number of interruptions and questions, and the handover moment.


  • Guideline adherence [MeSH]
  • Patient handoff [MeSH]
  • Emergency Medical services [MeSH]


Patient handover from one health care setting to another includes possible threats to quality and continuity of care [1]. A handover is characterized by the involvement of two or more professionals, the exchange of verbal and/or written information about the patient’s diagnosis, treatment and care, and the transition of patient responsibility [1-5]. The handover from ambulance to emergency department (ED) involves 2-way communication between the ambulance crew and ED-staff [6]. Especially the handover from ambulance to ED seems error prone as there is a high patient turnover, patients present themselves with a wide diversity of clinical conditions, there are acute time constraints, and the ambulance crew has only one opportunity to transfer patient information [7]. Previous studies report a loss of information during handover from ambulance to ED [5,8,9]. Factors which might influence the quality of the handover from ambulance to ED are a lack of active listening skills or inattention of ED-staff, unnecessary repetitions or provision of unnecessary information by ambulance crew, interruptions, workload, working relationships between ambulance crew and ED-staff, uncertainty about the transfer of responsibility, and failure to reach shared understanding [5,6,10-12].

To overcome these problems and barriers, standardization of the handover from ambulance to ED is recommended [5,13]. To facilitate standardization, structured models for ambulance to ED handover have been developed: (De)MIST (Demographics, Mechanism of Injury/illness, Injury or Illness found or suspected, Signs, Treatment given), AMPLE (Allergies, Medications, Past illnesses, Last meal, Events), ASHICE (Age, Sex, History, Injuries, Condition, Expected time of arrival), IMIST-AMBO (Identification of the patient, Mechanism/medical complaint, Injuries, Signs, Treatment and treatment respons/trend, Allergies, Medications, Background and Other), SOAP (Subjective information, Objective information, Assessment, Plan) and BAUM (‘Bestand’ (inventory), ‘Anamnese’ (medical history), ‘Klinische Untersuchungsergebnisse’ (clinical findings), ‘Massnahmen’ (actions)) [5,13-15].

To standardize handover practice from ambulance to ED in the Netherlands, an evidence-based guideline has been developed. The key-recommendation of the guideline is to use the DeMIST-model to structure prehospital notification and handover in the chain of ambulance, emergency medical dispatch (EMD) and ED. Due to a lack of evidence about effectiveness and applicability of handovers models, the choice for the DeMIST model was based on the fact that the MIST model was already in use. Other recommendations of the handover guideline are (a) that the pre-hospital professional who is responsible for the patient, provides a handover to the ED-professional who will be responsible for the patient, (b) that a handover takes place before patient transfer, and (c) that the ambulance crew verifies if the handover was clear.

To assist implementation of the newly developed guideline, a tailored e-learning program was developed to serve as educational intervention. The e-learning program was tailored to influencing factors that were identified beforehand in the local chain of emergency care. Previous studies show that e-learning can be effective to teach emergency physicians and nurses to recognize child abuse, to administer metoclopramide and to improve triage skills [16-18]. The effect of e-leaning on handover has not been studied.

Therefore, the aim of the study was to evaluate the effectiveness of an e-learning program to improve adherence to the handover guideline, hereby structuring pre-hospital notification and handover in the chain of ambulance-EMD-ED. We hypothesized that the e-learning was effective to improve handover on the two key-guideline recommendations (Table 1), being 1) to use the DeMIST model and 2) to use the DeMIST model in the correct sequence.
Table 1


Primary outcomes (N1 + N2 + O1)


Scoring options

Handover model used

Key-guideline recommendation


Correct sequence of DeMIST model

Key-guideline recommendation

Yes/no/Specification of sequence if incorrect

Secondary outcomes (O1)


Sender of the handover

Guideline recommendation

Ambulance nurse/ambulance driver

Composition of the receiving team

Guideline recommendation


Recognizability of the receiver

Guideline recommendation


Handover moment

Guideline recommendation

Before/during/after patient transfer

Verification if handover was clear

Guideline recommendation


Documentation of handover


Whiteboard/DeMIST-form/patient file/different

No. of clarifying questions asked by receiver


Actual number

No. of repetitions from sender


Actual number

No. of interruptions other than questions or repetitions


Actual number



The study had a prospective pre-test post-test design.


The study setting was located in the chain of emergency care of Nijmegen, the Netherlands. The chain of emergency care consists of the regional ambulance service (EMS), the emergency medical dispatch centre (EMD), and the emergency department (ED) of the Radboud university medical centre. In 2013, the EMD in this region managed 66.316 ambulance calls. Ambulances are staffed with one driver and one registered ambulance nurse, specialized EMD-nurses staff the EMD. Registered nurses become qualified as an ambulance nurse or EMD-nurse after following a specific national training course. Ambulance nurses work autonomously and are allowed to administer medical treatment based on their national protocol, without direct consultation of an EMS physician. The ED of the Radboudumc is a level 1 traumacenter, meaning the ED is delivering 24/7 trauma care for all types of patients. The ED had 21.672 admissions in 2012 and is staffed with emergency nurses and emergency physicians. Additional medical teams (trauma surgeon, intensivist) can be activated.

Handover process

The transfer process of the patient and patient information in the chain of emergency care is displayed in Figure 1. A request for an ambulance can be made by a lay person calling the national emergency number, or by another healthcare professional (e.g. general practitioner). The request is handled by the EMD-intake nurse, who interrogates the caller and triages the problem. During the intake, information is stored into (1) the EMD-system and (2) the digital ambulance run sheet. On the basis of this information and guided by dispatch-protocols, the EMD-dispatch nurse decides about dispatching an ambulance. The ambulance can be dispatched with urgency level A1 (arrival <15 minutes), A2 (arrival <30 minutes) or B (planned). After on-scene diagnosis and treatment, the ambulance nurse may decide to transfer the patient to the ED. The ambulance crew provides a notification (N1) to the EMD-dispatch nurse by telephone, then the EMD-dispatch nurse calls the ED and notifies the ED (N2). Both calls are logged into the EMD-system. At the same time, the ambulance sends a digital notification (N3) to the ED. The digital notification is a short version of the ambulance run sheet with DeMIST data. When the ambulance arrives at the ED, the handover of the patient, the patient information, and the patients’ personal belongings from ambulance to ED takes place (O1).
Figure 1
Figure 1

Patients transfer and handover.


The degree of adherence to the key-recommendation to use the DeMIST-model in the correct sequence to structure pre-hospital notification and handover from ambulance to ED was the primary outcome of this study for N1, N2 and O1 (Table 1). For handover (O1), secondary outcomes based on guideline recommendations were the professional providing a handover (sender), the composition of the receiving team, recognizability of the receiver, handover moment, and verification if the handover was clear. The choice for ‘composition of the receiving team’ was also based on the idea that if the receiving team is complete at the start of a handover, this reduces the risk of loss of information due to multiple handovers (‘Chinese whisper’) [5]. Additional secondary outcomes based on literature were documentation of information from the handover, the number of repetitions, number of questions, and the number of interruptions other than repetitions and questions.


The pre-test phase consisted of two periods of 4 weeks between April 22nd and August 9th 2013. Handovers from ambulance to ED for all types of ambulance runs and all types of patients were included. Handovers of patients with a possible or confirmed MRSA-contamination were excluded, as these handovers took place in separate rooms with infection precautions. Data were collected in two steps. First, all handovers from ambulance to ED were observed using a structured data collection form based on primary and secondary outcomes and scoring options (Table 1). Secondly, to collect data for the prehospital notification (N1 + N2) from ambulance to ED we used audiotapes. Corresponding audiotapes for a handover were identified at the EMD using an unique ambulance run identifier. All data were collected by trained 4th year students from the bachelor of nursing or bachelor of health studies.


We used Grol's model for effective implementation as study framework (Figure 1) [19,20]. The model provides a stepwise approach for improving clinical practice and starts with the identification of research findings or guidelines that have to be implemented (step 1). Steps 2 and 3 include a description of (change) targets and an analysis of the target group, current practice and setting. On the basis of this analysis, implementation strategies can be selected or developed (step 4), followed by the execution and evaluation of an implementation plan (steps 5 and 6). For step 1 in this study, the aim was to implement the handover guideline. The analysis of the target group (step 2 and 3) was undertaken composing a multi-disciplinary steering group and conducting a focus group interview. The multi-disciplinary steering group included opinion leaders from the chain of emergency-care: 1 EMD nurse, 1 ambulance nurse, 1 ambulance care medical manager, 1 emergency physician and 1 emergency nurse. The role of the steering group was to provide input in the study design and to create support for the study in the local chain of emergency care. The aim of the focus group interview was to identify handover problems in the local chain of emergency care. The focus group was organized in April 2013, participants were 2 emergency physicians, 2 emergency nurses, 2 ambulance nurses and 1 EMD-nurse. The focus group interview was audiotaped, and transcribed verbatim. Two researchers identified the problems on the basis of the transcription, and for each problem a key-determinant was added (Table 2). To enhance trustworthiness a member check was performed [21], therefore identified problems and determinants were sent to the participants of the focus group interview. Furthermore, problems which arose from the pre-test were also addressed.
Table 2

Handover problems identified in the chain of emergency care

Focus group interview


Non-usage of the DeMIST model

Knowledge, skills and motivation on:

Incorrect sequence of the DeMIST model

How to use the DeMIST for all types of patients (trauma and non-trauma)

Difficulties with applying the DeMIST model to trauma and non-trauma patients

The correct sequence of DeMIST

Handover of subjective information/interpretation of information (“patient is stable) instead of objective parameters

Usage of objective information

The timing of the handover

Ambulance crew has the impression that the digital notification is only used for retrieval of patient information and not for monitoring the patient

The documentation of handover

The advantages of using the DeMIST model in the chain of emergency care

Unclear for ambulance crew who is the receiver of the handover at the ED


77.9% of the handovers were structured with the DeMIST model

69.9% of the DeMIST handovers had the correct sequence

73.1% of the handovers took place after patient transfer

49% of the handovers were interrupted by questions from ED-staff

26.9% of the handovers were documented

On the basis of identified problems and determinants the steering group chose to use a tailored online e-learning program as intervention. Literature suggests tailoring interventions to identified problems to increase effectiveness, although the effectiveness of tailoring has not been proven irrefutable yet [22]. Reasons to choose e-learning were (a) the fact that all determinants could be addressed, (b) the flexibility, availability and accessibility of using e-learning which suits the emergency care context [23], and (c) the fact the target group was familiar with e-learning in their training programs. The e-learning program was specifically designed for EMD-nurses, ambulance nurses and drivers, emergency nurses and physicians, on the basis of (1) the handover guideline, (2) literature, (3) expert opinion, and (4) identified problems. The e-learning program yielded five components aimed at (1) knowledge about the DeMIST model and handover process, (2) skills how to use the DeMIST model to provide a proper handover, and (3) motivation to use the DeMIST model in the total chain of emergency care (Table 3).
Table 3

Components and content of the e-learning program





Explanation on the usage of the e-learning program and the learning goals to the participant

Learning goals

The professional knows the elements of a proper DeMIST handover

The professionals knows why it is important to use the DeMIST model for handover

The professional knows the role of the emergency medical dispatcher, ambulance driver, ambulance nurse, emergency nurse and emergency physician during handover

The professional performs a DeMIST handover for trauma and non-trauma patients


Provision of theory on DeMIST and its usage to the participant

Theory on DeMIST

De: full name, date of birth/age and sex of the patient

M: trauma or non-trauma

I: injuries found or suspected/complaints

S: Airway, Breathing (frequency, SpO2), Circulation (heart rate, blood pressure), and Disability (EMV-score, pupil reaction, pain, blood glucose)

T: working diagnosis, treatment given, effect of the treatment

When to use DeMIST

How to use DeMIST

Supply of objective information

When to provide a handover

Verification if handover was clear

Who provides a handover to whom (professionals’ role)

Advantages of using the DeMIST in the total chain of emergency care

Knowledge test

Summative test whether the participant has sufficient knowledge, insight and basic skills about a DeMIST handover

8 random questions out of 22 on knowledge, insight and application of theory

Simulation test

Summative test whether the participant can integrate knowledge and skills to provide a DeMIST handover

2 high fidelity simulation scenarios, randomly picked from 7 possible scenarios:

1. Female (75 yrs), low energetic trauma (pedestrian-car)

2. Male (45 yrs), high energetic trauma (tree-car)

3. Female (28 yrs), hypovolemic shock (fluxus post partum)

4. Male (30 yrs), fever, hypotension altered consciousness (septicaemia)

5. Female (55 yrs), resuscitation

6. Male (68 yrs), resuscitation

7. Female (70 yrs), stroke


Feedback on knowledge and simulation test to the participant

Achievement on learning goals with feedback

During the knowledge test, 8 random questions were presented. The caesura for the knowledge test was 87.5% (1 wrong answer). The e-learning program included 7 simulation scenarios on trauma (2x), resuscitation (2x), septicaemia (1x), fluxus post-partum (1x), and stroke (1x). All scenarios were designed with regard to the 3 diagnoses with the highest national incidence in emergency care: cardiology, trauma and internal medicine. The scenarios could be simulated from EMD, EMS or ED perspective that the professional could choose. This choice was added to the e-learning program to emphasize the chain of emergency care. Professionals could exercise simulations before entering the simulation test. During the simulation test the professionals had to simulate two scenarios (caesura 90%). The construction of the simulation test included that the result of the first scenario cumulated to the result of the second scenario, so if the professional failed the first simulation, zero faults were allowed during the second simulation. The rationale was that professionals provided a good handover the first time, as in real practice the ambulance crew has only one opportunity to provide a handover. Both the knowledge and simulation tests provided feedback to the professionals. The draft version of the e-learning program was tested on content and usability by representatives (n = 6) from the targetgroup. There was no maximum time restriction for completion of the entire e-learning program.

Intervention phase

All possible professionals from the EMD (n = 15) and EMS (n = 73) who could be involved in a handover during the study period were invited to follow the e-learning program on October 3rd 2013. The e-learning program was accompanied by an email in which the purpose was explained, and in which the professionals were motivated by their managers and educational coordinators. Also, information on the EMD and EMS intranet was published and professionals were motivated to follow the e-learning program by members of the steering group. The professionals could start the e-learning program any time on any computer they wanted, until November 16th 2013. During this period, each professional received 2 digital reminders. To stimulate the professionals to follow the e-learning program, the program was accredited with official registration points for EMD and ambulance nurses, and ambulance drivers.


The post-test phase lasted from November 11th until December 8th 2013. To collect data, the same methods were used as in the pre-test.

Data analysis

As this study is the first intervention study on handover, the number of observed handovers was based on feasibility and we did not perform a formal power analysis. To have an estimation, the minimum number of handovers required was determined by a power analysis using G*Power 3 [24]. Hereby, we set the α-level at .05 and the power level at .8. Based on these settings, we needed 143 handovers. Data were entered in SPSS and analysed using descriptive techniques. To compare the pre-test data with the post-test data, X2-tests and t-tests were performed. For all tests, statistical significance was set at P-value less than 0.05. To enhance validity and reliability, all handovers and audiotapes were observed/listened by two independent observers who discussed differences until consensus was reached. Inter-rater reliability was computed for a random sample of 10% (n = 17) of the observations in the post-test and was 91.9%.

Ethical considerations

On the basis of the study protocol, the Committee on Research Involving Human Subjects region Arnhem/Nijmegen waived the need for ethical approval (registration number 2013/046).



In total, 78/88 (88.6%) professionals followed the e-learning program, of which 70/78 (89.7%) certified for the knowledge test, and 41/70 (52.6%) certified for the simulation test also. The professionals spent an average median time on the e-learning program of 75 minutes (Table 4).
Table 4

Characteristics of the e-learning program


All professionals (n = 88)


n (%)

Started the e-learning program


78 (88.6)

Certification status of starters (n = 78)


n (%)

Knowledge test alone


29 (37.2)

Knowledge test + simulation test


41 (52.6)

Started but no certification


8 (10.3)

Average time spent of starters (n = 78)

25th percentile


75th percentile

Time spent on theory (in minutes)




Time spent on knowledge test (in minutes)




Time spent on simulation test (in minutes)




Total time spent (in minutes)




Handover from ambulance to ED

All observed handovers during the pre-test (n = 145) were included. From the observed post-test handovers (n = 169), two handovers were excluded as these were provided by EMS-students who did not work at the EMS during the intervention period, leaving a total of 167 handovers. There were no significant differences between both study periods regarding patient gender, medical specialty or urgency.

Regarding the primary outcome, no significant difference in the number of handovers that were structured with the DeMIST model between the pre-test (77.9%) and the post-test (73.1%) existed (Table 5). In the pre-test, 69.9% of the DeMIST handovers used a correct sequence, in the post-test this was 70.5%. When professionals deviated from the correct sequence during both the pre- and post-test, the most common deviation was to mix or switch the ‘S’ and ‘T’. There was no association between medical specialty (trauma/non-trauma) and the correct sequence (X2 = .36, p = .872).
Table 5

Handover from ambulance to ED (O1)


Pre-test (n = 145)

Post-test (n = 167)

p-value X2-test

n (%)

n (%)



Patient gender




75 (51.7)

83 (49.7)



70 (48.3)

73 (43.7)




10 (6.0)


Medical specialty




32 (22.1)

25 (15.0)



113 (77.9)

142 (85.0)




A1 (within 15 minutes)

49 (33.8)

44 (26.3)


A2 (within 30 minutes)

81 (55.9)

93 (55.7)


B (low urgency/planned)

15 (10.3)

30 (18.0)


Primary outcomes


Handover model used



113 (77.9)

122 (73.1)



0 (0.0)

2 (1.2)



0 (0.0)

1 (0.6)


No method/not recognizable

32 (22.1)

42 (25.1)


Correct sequence of the DeMIST



79 (69.9)

86 (70.5)



34 (30.1)

24 (19.7)


No sequence recognizable within DeMIST


12 (9.8)


Secondary outcomes


Receiving team composition at start handover


Physician and nurse

62 (42.8)

68 (40.7)


Physician later than start

64 (44.1)

90 (53.9)


Nurse later than start

18 (12.4)

9 (5.4)


Physician and nurse too late

1 (0.7)



Receivers recognizability



33 (22.8)

39 (23.4)



29 (20.0)

49 (29.3)


Not recognizable

83 (57,2)

79 (47.3)


Handover given by


Ambulance nurse

143 (98.6)

157 (94.0)


Ambulance nurse + ambulance driver

2 (1.4)

10 (6.0)


Receiver handover


ED physician

19 (13.1)

26 (15.6)


ED nurse

64 (44.1)

89 (53.3)


ED team(minimum: ED nurse + ED physician)

61 (42.1)

52 (31.1)



1 (0.7)



Handover moment


Before patient transfer

28 (19.3)

40 (24.0)


During patient transfer

3 (2.1)

9 (5.4)


After patient transfer

106 (73.1)

110 (65.9)


Different (patient is to toilet or is in different room)

8 (5.5)

8 (4.8)


Number of handovers with repetitions

17 (11.8)

21 (12.6)


Number of handovers with questions

71 (49.0)

115 (68.9)


Number of handovers with interruptions

22 (15.2)

88 (52.7)


Number of handover were verification was asked

22 (15.2)

19 (11.4)


Handover documented

39 (26.9)

49 (29.3)



0 (0.0)

2 (4.1)


DeMIST form

1 (2.6)

3 (6.1)


Patient file

0 (0.0)

1 (2.0)



38 (97.4)

43 (87.8)


*The gender was not registered for 10 patients, this could not be retrieved.

**Significant difference.

***Glove, napkin, sheet, paper.

The composition of the receiving team differed after the intervention as less handovers started with presence of an emergency physician. There was no difference in how the receivers made themselves recognizable. During the post-test, significantly more handovers in which the ambulance driver was involved took place. Between pre- and post-test, there were no significant differences between the moment of the handovers, although the highest number of handovers take place after patient transfer. After the intervention, the number of handover with questions (p = .000) and interruptions (p = .000) significantly increased, the number of handovers with repetitions did not differ. The percentage of handover with a verification did not significantly decrease.

Pre-hospital notification

During the pre-hospital notification from ambulance to EMD (N1), no significant difference in the number of handovers that were structured with the DeMIST model between the pre-test (72.9%) and post-test (80.7%) existed (Table 6). In the pre-test, 66.7% of the DeMIST handovers used a correct sequence, in the post-test this was 56.5%. During the pre-hospital notification from EMD to ED (N2), no significant difference in how many handovers were structured with the DeMIST model between the pre-test (83.3%) and post-test (86.5%) existed. In the pre-test, 84.0% of the DeMIST handovers used a correct sequence, in the post-test this was 73.3%.
Table 6

Prehospital notification (N1 + N2)


Pre-test (n = 145)

Post-test (n = 167)

p-value X 2 -test

n (%)

n (%)

EMS to EMD by telephone (N1)


Notification given

70 (48.3)

57 (34.1)


Handover model used



51 (72.9)

46 (80.7)



1 (1.4)




1 (1.4)



No method/not recognizable

17 (24.3)

11 (19.3)


Correct sequence of the DeMIST



34 (66.7)

26 (56.5)



17 (33.3)

20 (43.5)


EMS to ED by telephone (N2)


Notification given

60 (41.4)

52 (31.1)


Handover model used



50 (83.3)

45 (86.5)



1 (1.7)







No method/not recognizable

9 (15.0)

7 (13.5)


Correct sequence of the DeMIST



42 (84.0)

33 (73.3)



8 (16.0)

12 (26.7)



This study evaluated the effectiveness of a tailored e-learning program to improve adherence to a handover guideline in the chain of emergency care. A total of 314 handovers from ambulance to ED were observed and results show no significant differences regarding the usage and correct sequence of the DeMIST model between the pre-test and post-test.

Results from both the pre-test and post-test phase show adherence rates to the DeMIST model ranging from 77.9%-73.1%, and adherence rates for correct sequence ranging from 69.9%-70.5%. To our knowledge, no studies investigated adherence to an ambulance to ED handover model in real practice, only one study assessed adherence the ISBAR handover model in a simulated setting, reporting an improvement in correct sequence from 0%-46% after a high-fidelity simulation intervention [25]. Compared to other guideline adherence rates in the prehospital and ED setting, adherence in our study is relatively high [26]. Nevertheless, the results also indicate room for improvement as in 22.1%-25.1% of the handovers no model was recognizable. This might incorporate the risk for loss or deformation of essential information. Possibly, this result indicates that professionals might perceive that the DeMIST model does not fit entirely for all patients handed over from ambulance to ED. For instance, one often heard counter argument for the (De)MIST is that it might be less applicable to non-trauma or non-critical patients [13], however our results show no association between trauma or non-trauma and the correct sequence of the DeMIST. On the other hand, one might argue that there are no valid reasons to deviate from a handover guideline, in contrast to diagnostic or therapeutic guidelines and protocols where deviations on the basis of patient conditions or preferences can be justified.

The e-learning program was not effective in improving and thereby implementing the new guideline, this can be explained by several reasons. The first reason might be the relatively high baseline adherence rates. These high rates can be caused by the (De)MIST integration in basic emergency care education in the Netherlands. Another reason might be the sole use of the e-learning program as the sole use of an educational intervention might not be effective [20]. However, emergency care research shows moderate to good effects of the sole use of e-learning [16-18]. Furthermore, our results might also urge the use of blended-learning were e-learning is combined with face-to-face educational meetings [23]. Despite these results, the effectiveness of e-learning should be further investigated as it is widely used to educate and train emergency care professionals [23].

A third reason might be that only the handover senders (EMD and ambulance professionals) were trained. During the study period it was not possible to train the ED-staff because they already were in training for the implementation of a digital patient file. A previous study showed that information retention by ED-staff decreased from 56.6% to 49.2% if the handover model is implemented in the ambulance setting only [7]. This stresses the need to implement a handover model in the chain of emergency care.

A fourth reason is the fact that 88.6% of the professionals started the e-learning program and that 52.6% of the starters certified for the whole program. This means that the intervention did not fully reach all intended professionals. The variation around the median time spent on the simulation test reflects the struggle professionals had with certifying for the simulation test, which was due to the accumulation of the result of scenario 1 with scenario 2. This accumulation resulted in a significant amount of the professionals who had to try several times before certifying for the simulation test which caused the high variation, and in 37.2% of the professionals who only certified for the knowledge test.

Despite relatively high adherence rates for the key-guideline recommendation, our results indicate several areas of improvement for handover from ambulance to ED. Firstly, in 26.9%-29.2% of the handovers transferred information was documented by ED-staff during handover. Most information was written down on gloves, napkins, pieces of paper or sheets, which carries the risk that this information is not integrated in medical records. In our study medical records were not checked for documented information after handover, but previous studies indicate suboptimal documentation of transferred information [9,27].

Secondly, the number of handovers in which verification was asked by ambulance staff, dropped from 15.2% to 11.4%. A previous simulation study also showed low rates of verification, although these rates increased after simulation exercises [25]. Verification of a handover indicates the end of the handover and might prevent interruptions of the handover due to questions asked by ED-staff.

Thirdly, most handovers took place after patient transfer in the ED. Handing over a patient during or after transfer, incorporates the risk that ED-staff already starts diagnostic or therapeutic actions that might distract ED-staff from the handover.

Fourthly, in 44.1%-53.9% the complete team was not present at the start of the handover. A previous study reported physician absence at 88% of the handovers [28]. Our results might be caused by lower-acuity patients for whom it is less urgent to be seen by a physician. Another explanation might be that there is no pre-hospital notification given by the ambulance crew to ED, as with 41.4% of the handovers in the pre-test and 31.1% of the handovers in the post-test a verbal notification was given. Another reason can be that the digital pre-hospital notification (N3) arrives too late at the ED sometimes, and the handover already took place.

Fifthly, the number of handovers with questions from ED-staff and interruptions significantly increased after the intervention. This might be caused by the fact that only the senders of the handover were trained, and that the receiving ED-staff had to get used to the structure. Most of the questions were related to the vital signs (‘S’) and treatment (‘T’). The treatment given is marked by emergency nurses and physicians as an essential element of the handover [6], this might explain the questions. In this study we were not able to mark the questions and repetitions as contributing to the handover or disturbing the handover. Repetitions and questions might contribute to the handover as they can clarify treatment, and lead to hearing specific aspects of the handover again [6]. On the other hand, repetitions and questions might disturb a handover as they might reflect a lack of listening skills or inattention of ED-staff, and ambulance staff gets frustrated if they have to repeat themselves, or as their findings are questioned [6,13]. Most of the interruptions were related to the patient or patients' next of kin. There were relevant interruptions (changing or adding information), and non-relevant interruptions which that were caused by the patient talking (with next of kin or the ambulance driver), phones ringing, and the arrival of other professionals.

Strengths and weaknesses

Obviously, the absence of a control group might be a threat to the external validity. Another threat to the validity of this study is the Hawthorne effect: the ambulance crew and ED personnel could see the observers when they were present at the ED. Furthermore, not all staff participated in the full intervention, which could explain limited effects. Also, it is possible there are other determinants that influence handover which were not integrated in the e-learning program, making the e-learning program less powerful. Inter-rater reliability between 2 observers was calculated for 10% of the observations, showing a satisfying 91.9% agreement. To increase reliability between pre- and post-test, observers of the post-test were trained by observers from the pre-test, but despite this effort slight differences in observations between pre- and post-test might have occurred. Finally, statistical significance may have occurred due to multiple X2-testing, although in the light of the low number of significant tests this did not seem a major issue.

Future research

Future research should focus on the applicability of different handover models to structure the handover in the chain of emergency care. Which models are applicable for which settings and patients groups? Also, the additional effect of training the receivers of the handover (ED-staff) should be investigated. Furthermore, the use of multiple strategies or blended learning should be examined for their effectiveness to improve handover practice. Finally, the user satisfaction of e-learning to implement a (handover) guideline can be evaluated.


This pre-test post-test study found no effect of a tailored e-learning program on adherence to a handover guideline. The results suggest that e-learning alone does not improve adherence. Despite the relatively high baseline adherence, our results indicate room for improvement in the handover process, with regard to documentation of information during the handover, the handover moment, and the completeness of the receiving team at the start of the handover.



The authors would like to thank Wim Huijzendveld and Paul Sanders for their role in motivating the professionals to follow the e-learning program. Furthermore, the authors would like to thank all the students from the bachelor nursing and bachelor of health for their support during data collection.

Authors’ Affiliations

Research department acute care, HAN University of Applied Sciences, Nijmegen, the Netherlands
Ambulance Service Gelderland Zuid, Veiligheidsregio Gelderland Zuid, Nijmegen, the Netherlands
Emergency Department, Radboud university medical center, Nijmegen, the Netherlands
Centre for Health Services and Nursing Research, KU Leuven, Leuven, Belgium
Scientific Institute for Quality of Healthcare, Radboud university medical center, Nijmegen, the Netherlands
Canisius Wilhelmina Hospital, Nijmegen, the Netherlands


  1. Benham-Hutchins MM, Effken JA. Multi-professional patterns and methods of communication during patient handoffs. Int J Med Inf. 2010;79:252–67.View ArticleGoogle Scholar
  2. Bruce K, Suserud BO. The handover process and triage of ambulance-borne patients: the experiences of emergency nurses. Nurs Crit Care. 2005;10:201–9.View ArticlePubMedGoogle Scholar
  3. Cheung DS, Kelly JJ, Beach C, Berkeley RP, Bitterman RA, Broida RI, et al. Improving handoffs in the emergency department. Ann Emerg Med. 2010;55:171–80.View ArticlePubMedGoogle Scholar
  4. Lawrence RH, Tomolo AM, Garlisi AP, Aron DC. Conceptualizing handover strategies at change of shift in the emergency department: A grounded theory study. BMC Health Serv Res. 2008;8:256. doi:10.1186/1472-6963-8-256Google Scholar
  5. Jensen SM, Lippert A, Ostergaard D. Handover of patients: a topical review of ambulance crew to emergency department handover. Acta Anaesthesiol Scand. 2013;57:964–970 10.1111/aas.12125 [doi].Google Scholar
  6. Jenkin A, Belson-Mitchell N, Cooper S. Patient handover: time for a change? Accid Emerg Nurs. 2007;15:141–7.View ArticlePubMedGoogle Scholar
  7. Talbot R, Bleetman A. Retention of information by emergency department staff at ambulance handover: do standardised approaches work? Emerg Med J. 2007;24:539–42.View ArticlePubMed CentralPubMedGoogle Scholar
  8. Carter AJE, Davis KA, Evans LV, Cone DC. Information loss in emergency medical services handover of trauma patients. Prehosp Emerg Care. 2009;13:280–5.View ArticlePubMedGoogle Scholar
  9. Evans SM, Murray A, Patrick I, Fitzgerald M, Smith S, Andrianopoulos N, et al. Assessing clinical handover between paramedics and the trauma team. Injury. 2010;41:460–4.View ArticlePubMedGoogle Scholar
  10. Bost N, Crilly J, Patterson E, Chaboyer W. Clinical handover of patients arriving by ambulance to a hospital emergency department: a qualitative study. Int Emerg Nurs. 2012;20:133–141 10.1016/j.ienj.2011.10.002; 10.1016/j.ienj.2011.10.002.Google Scholar
  11. Owen C, Hemmings L, Brown T. Lost in translation: maximizing handover effectiveness between paramedics and receiving staff in the emergency department. Emerg Med Australas. 2009;21:102–107 10.1111/j.1742-6723.2009.01168.x; 10.1111/j.1742-6723.2009.01168.x.Google Scholar
  12. Thakore S, Morrison W. A survey of the perceived quality of patient handover by ambulance staff in the resuscitation room. Emerg Med J. 2001;18:293–6.View ArticlePubMed CentralPubMedGoogle Scholar
  13. Bost N, Crilly J, Wallis M, Patterson E, Chaboyer W. Clinical handover of patients arriving by ambulance to the emergency department - a literature review. Int Emerg Nurs. 2010;18:210–220 10.1016/j.ienj.2009.11.006; 10.1016/j.ienj.2009.11.006.Google Scholar
  14. Riesenberg LA, Leitzsch J, Little BW. Systematic review of handoff mnemonics literature. Am J Med Qual. 2009;24:196–204 10.1177/1062860609332512.View ArticlePubMedGoogle Scholar
  15. Iedema R, Ball C, Daly B, Young J, Green T, Middleton PM, et al. Design and trial of a new ambulance-to-emergency department handover protocol: 'IMIST-AMBO'. BMJ Qual Saf. 2012;21:627–633 10.1136/bmjqs-2011-000766; 10.1136/bmjqs-2011-000766.Google Scholar
  16. Rankin JA, Then KL, Atack L. Can emergency nurses' triage skills be improved by online learning? Results of an experiment. J Emerg Nurs. 2013;39:20–26 10.1016/j.jen.2011.07.004; 10.1016/j.jen.2011.07.004.Google Scholar
  17. Smeekens AE, Broekhuijsen-van Henten DM, Sittig JS, Russel IM, ten Cate OT, Turner NM, et al. Successful e-learning programme on the detection of child abuse in emergency departments: a randomised controlled trial. Arch Dis Child. 2011;96:330–334 10.1136/adc.2010.190801.Google Scholar
  18. Yeoh BS, Taylor DM, Taylor SE. Education initiative improves the evidence-based use of metoclopramide following morphine administration in the emergency department. Emerg Med Australas. 2009;21:178–183 10.1111/j.1742-6723.2009.01179.x.Google Scholar
  19. Van Achterberg T, Schoonhoven L, Grol R. Nursing implementation science: how evidence-based nursing requires evidence-based implementation. J Nurs Scholarsh. 2008;40:302–10.View ArticlePubMedGoogle Scholar
  20. Grol R, Wensing M, Eccles M. Improving Patient Care. The Implementation of Change in Clinical Practice. Toronto: Elsevier Science; 2005.Google Scholar
  21. Holloway I, Wheeler S. Qualitative reseach in nursing. Oxford UK: Blackwell Science ltd; 2004.Google Scholar
  22. Grol R, Wensing M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004;180:S57–60.PubMedGoogle Scholar
  23. Roe D, Carley S, Sherratt C. Potential and limitations of e-learning in emergency medicine. Emerg Med J. 2010;27:100–104 10.1136/emj.2008.064915.Google Scholar
  24. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–91.View ArticlePubMedGoogle Scholar
  25. Dojmi Di Delupis F, Pisanelli P, Di Luccio G, Kennedy M, Tellini S, Nenci N, et al. Communication during handover in the pre-hospital/hospital interface in Italy: from evaluation to implementation of multidisciplinary training through high-fidelity simulation. Intern Emerg Med. 2014 10.1007/s11739-013-1040-9.Google Scholar
  26. Ebben RH, Vloet LC, Verhofstad MH, Meijer S, Mintjes-de Groot JA, van Achterberg T. Adherence to guidelines and protocols in the prehospital and emergency care setting: a systematic review. Scand J Trauma Resusc Emerg Med. 2013;21:9-7241-21-9 10.1186/1757-7241-21-9; 10.1186/1757-7241-21-9.Google Scholar
  27. Murray SL, Crouch R, Ainsworth-Smith M. Quality of the handover of patient care: a comparison of pre-Hospital and Emergency Department notes. Int Emerg Nurs. 2012;20:24–27 10.1016/j.ienj.2010.09.004; 10.1016/j.ienj.2010.09.004.Google Scholar
  28. Yong G, Dent AW, Weiland TJ. Handover from paramedics: observations and emergency department clinician perceptions. Emerg Med Australas. 2008;20:149–155 10.1111/j.1742-6723.2007.01035.x.Google Scholar


© Ebben et al.; licensee BioMed Central. 2015

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.