Effects of pre-training using serious game technology on CPR performance – an exploratory quasi-experimental transfer study

Recruitment and sample

This study was designed as an exploratory quasi-experimental controlled study with 3 groups: Two groups had attended MVW-CPR training, 18 and 6 months respectively, before the study (18 m and 6 m), and one group served as reference group (control). The latter group was matched to the other groups, consisting of peers attending the same semester of medical school. Thirty-six medical students at Karolinska Institutet volunteered to participate during their preclinical period (second and third year). All of them had attended a compulsory conventional manikin based CPR training course during their first semester of medical school. Subjects were enrolled by announcements. The study was approved by the regional ethics committee at Karolinska Institutet, and informed consent was obtained from the participants.

MVW-CPR team training

The first phase of the study consisted of MVW-CPR team training. The training comprised two sessions. The first session started with a 10 minute rehearsal lecture on basic life support (BLS), followed by an approximately 20 minute long familiarization to the virtual environment. During this the participants learned how to control the avatar using the keyboard, communicate with each other using a headset with microphone, and performing various tasks. The actual team MVW-CPR training consisted of 4 short (4-5 minutes) scenarios. In these the subjects in groups of 3 had to take care of a cardiac arrest victim that collapsed in front of them. This involved approaching the victim, examining the victim and starting resuscitation as stated by the guidelines. The latter included an emergency phone call to the 911 dispatcher. The actions had to be performed in collaboration within the group. Following each scenario a brief (3-5 minute) feed-back session followed. Standard personal computers were used which were connected to the virtual world by broad band internet connections. After 6 months the subjects attended a similar session without the lecture.

Loss of subjects

Assessments and further training

At start a pre-test on knowledge of basic life support was given. A post-test was performed after training. The knowledge tests consisted of 10 true/false statements. Two versions of the test existed in order to let each subject have different versions before and after training (provided as Additional file1). Half of the subjects in each group started with one version, the rest with the other, after training this was reversed. Three or 6 items (depending on version) were directly related to bystander-CPR. Scores on the test were between 0-10, scores on CPR specific part were graded between 0-100 percent correct.

Before assessment of CPR skills the subjects were presented with a short (approx. 10 min) standardized lecture on BLS, including bystander-CPR in order to give a common theoretical base. Also a 5 minute familiarization to the SimMan® full-scale simulator (Laerdal Medical) and the environment (in particular how to call for help), was included.

The subjects were assessed and trained in teams of 2 or 3 rescuers. The first scenario was an assessment scenario. It was followed by a 20 minute standardized lecture on effective teamwork introducing the participants to so called Crew Resource Management (CRM) principles[29]. Thereafter training scenarios ensued during which two teams took turns in performing and observing. All over each team trained in 3 scenarios. In two of the training scenarios start of CPR was expected as required by the guidelines. In the third training scenario start of CPR was contra-indicated. The fifth and last scenario was again an assessment scenario. Figure1 summarizes the design of the study.

During the scenarios the teams had to assess and resuscitate a victim by following the bystander-CPR guidelines (2005 version)[30]. At approximately 7 minutes after the yell for help the scenario was ended by help of a paramedic arriving at the scene, taking over responsibilities. After each scenario feed-back was given by an instructor and, when present, by observers. The feed-back focused on the CPR guidelines as well as on aspects of teamwork.

Analysis and statistical methods

All scenarios were video-taped for later analysis of CPR performance. This analysis was performed by a trained blinded assessor. Several measurements of performance were used: On group level time between the rescuers entering the room until start of chest compressions was calculated and no-flow time was determined. This was calculated as the total time that elapsed between the cycles of chest compression divided by the number of cycles performed minus 1. Furthermore adherence to the CPR guidelines was scored (provided as Additional file2). On individual level the frequency of chest compressions was calculated. Friedman repeated measures analysis of variance on ranks was used for time dependent non-parametric data. Statistical comparisons, before-after, were made by use of the Wilcoxon signed rank test. Comparisons between groups were made by ANOVA on ranks with pairwise comparisons using Dunn’s method. The test on CPR-specific knowledge was performed as a post-hoc analysis. For nominal data Chi square test was used. The significance level was set at P < 0.05. As P values are not adjusted for multiple testing, they have to be considered as descriptive. The calculations were performed using SigmaStat version 3.5 (Systat Software Inc, Point Richmond, CA, USA). Data are presented as mean [±SD].

Knowledge

The change of overall knowledge and CPR-specific knowledge in the entire study group changed from 7.1 (±1.5) to 8.3 (±0.9) (p < 0.001) and 78% (±24%) to 95% (±10%) (p < 0.001), respectively.

Performance

Time from subjects entering the room until start of chest compressions did not differ between groups (during first scenario: 18 months group 23 (±11) seconds, 6 months group 21 (±7) seconds and control group 24 (±13) seconds respectively) or change significantly over time. The no-flow time (time after start of CPR, where no chest compressions were administered) did not differ significantly between groups (during first scenario: 18 months group 7.9 (±2.0) seconds, 6 months group 9.4 (±2.5) seconds and control group 7.8 (±1.4) respectively, during the last scenario: 18 months group 7.5 (±0.8) seconds, 6 months group 6.8 (±1.0) and control group 6.9 (±1.8) seconds respectively). During video analysis it was evident that this variable was rather complex; if for example only one rescue breath was given, or if they were given to rapidly the no-flow time would be shorter, but at the same time the guidelines were violated.

There was a clear difference in the first scenario regarding adherence to the CPR guidelines (Figure2). This included performing the actions in a non-prescribed order, excluding diagnostic steps, stopping to check for pulses and not performing 30:2 cycles stated by the CPR guidelines. This difference was not detectable during any of the following scenarios. We also noticed a tendency for less compliance with the guidelines during the last scenario for all groups.

There was also a significant difference in the number of chest compression cycles that did not agree with frequencies stipulated by the CPR guidelines during the first scenario. In the 18 months group 44 (±49)% of the compression cycles occurred at frequencies of less than 80 /min or more than 125 /min. For the 6 months and control group these values were 0% and 54 (±44)% respectively. The difference between the control group and the 6 months group was statistically significant (p < 0.001), as was the difference between the 18 months group and the 6 months group (p < 0.001). Overall, as training progressed, there was a tendency towards increased compression rates. Also, in general, the large variation of frequencies seen in the 18 months group and the control group decreased. In particular in the 6 months group the increase over time of the compression frequency led to a large proportion of cycles with frequencies over 125 /min. In the last scenario there was significantly less incorrectly paced cycles in the control group compared to the 6 months group (Figure3).

CPR knowledge

Gaps in CPR related knowledge between the groups before training and in performance during the first scenario indicated that knowledge and behaviors trained in the virtual setting were retained to some degree in contrast to lectured knowledge where no retention could be detected. After having trained in 5 scenarios, the knowledge gap in CPR related knowledge was not detectable, and the performance gap in the last scenario had disappeared.

Evidently, the rationale for CPR training is preparing potential rescuers to actually perform CPR in real life. In this respect theoretical knowledge is useful only if applied correctly. It can be argued that theoretical and correctly applied knowledge are connected[24]. Better knowledge in theory supports correct action and might provide a scaffold for future learning. Factual knowledge may also provide basis for efficacy beliefs and create a more complete mental model of the phenomenon to promote retention of procedural knowledge.

Time before start of CPR

During real-world CPR, time is a factor of crucial importance and was consequently recorded. As shown by the large variation, in this standardized setting, this parameter was difficult to draw any conclusions from. When assessing the recordings of the scenarios for performance, we also noted a large variation in competitiveness and acting amongst the participants which in turn might have concealed actual differences in how effective the initiation of CPR was.

Adherence to CPR guidelines

Although, due to the small sample size in this exploratory study, statistical methods have not been used on this data, support for pre-training was found in how well the CPR algorithm, stated by the bystander CPR guidelines, was followed. Virtual world pre-training reduced uncertainty and initiatives outside the defined algorithm. Interestingly, this clear pattern was found, despite the fact that all subjects just minutes earlier had attended a standardized, brief rehearsal lecture. As was hypothesized, this indicated that actual training – although carried out several months earlier in a virtual world – would still outweigh the benefits of the lecture. Hence, probably memory of subject knowledge alone is not the key to correct CPR performance.

Psychomotor issues

Among individual psychomotor performance measures, compression frequency and quality seem to be among the most important ones for real life positive outcome. During the first scenario there was a huge variation in frequency in the 18 months and control groups. Many of these subjects were outside the stipulated frequency range. After feed-back and recurrent training this variation was considerably smaller. Notably the subjects in the 6 months group were “right on spot”, but as training progressed, showed a drift towards too high rates. This may indicate that as training goes on the participants may get overly vigorous and that more focus might be needed on this phenomenon, alternatively support the use of technical or cognitive tools for correct psychomotor output keeping frequency of compressions correct[31–33].

Another important parameter is the “no-flow time”[34, 35]. In this study it was defined as the time, after start of CPR, where no chest compressions were administered. Although calculated during all scenarios, this time did not change much during the training. The lowest no-flow time was recorded during scenarios where there were several quality issues, such as reduced quality in the rescue breaths. Therefore we did not further consider this data.

New CPR training methods have been developed and refined as reactions to several problems in conventional CPR training. In a previous study we argued that the administered MVW-CPR cannot stand alone as means of CPR training[26]. However, as indicated in this exploratory study, the use of MVW-CPR pre-training might be one way of preparing students for conventional CPR courses. This added training may serve as cognitive support for future training and real-world tasks, and is in line with other trends in medicine[36]. Possibly MVW-CPR team training can also be used for rehearsal after other modes of CPR training, although it has not been the focus for the current study.

In this study the trainees are digital natives. Although CPR skills are important in many age groups, we believe that it would be easier to implement MVW-CPR team training in this group. The aged population, in which cardiac arrests most commonly occur, might also possibly benefit from such training, however from this study we can draw no such conclusions and results from such training may be much different than current results. For other digital native groups we expect that the positive effect of MVW-CPR team training would be comparable.

The strength of this study is its standardization regarding participants and protocol. CPR is often performed in groups[5]. Virtual world CPR team training with avatars has the advantage to address this issue. Although also assessments in this study to some extent were on group level, this introduced a weakness because group assessment carries the problem of decreasing statistical power. Further, the performance of groups is to some extent dependent on teamwork skills which can differ between different group constellations. One way of trying to reduce such influences was by adding a lecture on teamwork to the training program. Another limitation of this study is its small sample size. The study was conceived as a follow-up on subjects that had already received virtual CPR training. The limited sample was further reduced by loss of subjects. We therefore stress the exploratory nature of the study and must hence remember that the results have to be interpreted with caution. The validity of assessing CPR skills by use of full-scale simulators can also be questioned. To our knowledge no studies on this issue have been performed. On the other hand, this technique for training and assessment is already implemented for clinical training of skills and behaviors in various settings and scenarios[37].

A focus of this training study was to standardize the scenarios in order to evaluate how pre-training and subsequent full-scale training affected the participants. Full-scale simulator team training is costly and in very limited supply, and therefore cannot be seen as a realistic future alternative for larger groups in society. In this study, in the absence of real-world alternatives, full-scale simulation was mainly used as a way of creating a reasonably realistic way of assessing transfer.

Serious games for learning and training are receiving much attention. However, transfer of CPR skills trained in serious games is difficult to test in authentic CPR. Thus, this exploratory quasi-transfer study was designed to probe the potential future use for virtual worlds in emergency medicine.