Clinical Investigations External Validation of Multimodal Termination of Resuscitation Rules for Out-of-hospital Cardiac Arrest Patients in the Covid-19 Era

Background:Futile resuscitation for out-of-hospital cardiac arrest (OHCA) patients in the COVID-19 era can lead to risk of disease transmission and unnecessary transport. Various existing basic or advanced life support (BLS or ALS) rules for termination of resuscitation (TOR)have been derived and validated in North America and Asian countries. This study aimed to evaluate the external validation of these rules in predicting the survival outcomes of OHCA patients in the COVID-19 era. Methods: A multicenter observational study was performed using the WinCOVID-19 Daegu registry data collected from 18 February to 31March 2020. The outcomes of each rule were compared to the actual patient survival outcomes. The sensitivity, specicity, false positive ratio (FPR), and positive predictive value (PPV) of each TOR rule were evaluated. Results: Of the 184 OHCA patients, 170 patients,who showed cardiac arrest of presumed cardiac etiology, were enrolled. TOR was recommended for 122 patientsbased on the international BLS-TOR rule, which showed 85% specicity, 74% sensitivity, 0.8% FPR, and 99% PPV for predicting unfavorable survival outcomes. When the traditional BLS-TOR rules and KoCARC TOR rule II were applied to our registry, one patient met the TOR criteria but survived at hospital discharge. With regard to the FPR (upper limit of 95% condence interval<5%) and PPV (>99%) criteria, only the KoCARCTOR rule I, which included a combination ofthree factors including not being witnessed by emergency medical technicians, presenting with an asystole at the scene, and not experiencing prehospital shock delivery or ROSC, was found to be superiorto all other TOR rules. Conclusion: Among the previous nine BLS and ALS TOR rules, KoCARCTOR rule I was most suitable for predicting poor survival outcomes and showed improved diagnostic performance. Further research on variations in resources and treatment protocols among facilities, regions, and cultures will be useful in determining the feasibility of TOR rules for COVID-19 patients worldwide.


Background
We are still ghting the novelcoronavirusdisease  at the forefront worldwide [1].Due to the symptoms and complications that individuals presentwith and transmission e ciency of COVID-19, we changedour standard treatment processes and personal protection strategies. In particular, resuscitation of cardiac arrest patients posed an especially high risk of transmissionof disease to healthcare professionals [2,3].For in-hospital cardiac arrest (IHCA) patients, this procedure wastypically performedafter COVID-19 had been con rmed, but for out-of-hospital cardiac arrest (OHCA) patients, resuscitationwas performed prior to any testing or con rmation. Consequently, the OHCA group was especially vulnerable.
In the prehospital setting, emergency medical personnel screened patients for COVID-19 based on their symptoms (fever, dyspnea, cough, and sore throat) and inquired about any recent historyof contact with infected individuals. However,this method was not always ideal because many carrierswere asymptomatic [4,5].As such, personal protective equipment (PPE) and medical resources in intensive care units (ICUs)were not alwaysutilized by physicians who tended to COVID-19 patients, andpatients who failed to experience a return of spontaneous circulation (ROSC) in the event of a cardiac arrest were often not transferred to a hospital due to the low likelihood of their survival [6]. Fortunately, recent cardiopulmonary resuscitation (CPR) guidelineswere changed, recommendingthat all healthcare providers assess the safety of the scene and wear higher levels of PPE for protection against infectious airborne dropletsand detailing the criteria for the basic life support termination of resuscitation (BLS-TOR) rule. These guidelines stated that before terminating resuscitative efforts, the healthcare workers must check that 1) the arrest was not witnessed byemergency medical service (EMS) personnel or a rst responder,2)ROSC was not observed prior to the patient being transported to the emergency medical centre (EMC), and 3) no electrical shocks were delivered [7][8][9][10][11][12][13][14].
We applied the rules that were typically selected depending on the country or region where the derivation and validation phaseswere conducted.These included the(1) International BLS and advanced life support (ALS) rules derived and validated in the United States and Europe [8], (2) Goto and KANTO-SOS rules developed in Japan and Asian countries [13,14], and (3) Korean OHCA registry-based TOR models,Korean Cardiac Arrest Research Consortium (KoCARC) TOR rules, and two new TOR models, whichwere used in our previous studies [7,11].
In Daegu metropolitan city, the number of patients with COVID-19 increased rapidlyat the community level fromFebruary 18, 2020, in response to the pandemic outbreak in South Korea.During its 7-week peak, we resuscitated 184 OHCA patients at six EMCs. This was the rst study to determine the e cacy of these TOR guidelinesfor an emerging infectious disease [7,11,12].

Enrolled OHCA patients
Daegu is a metropolitan city with a population of 2.44 million people, and is composed of two regional level I EMCs, four local level II EMCs, and 19 emergency facilities or clinics. In particular, there are six EMCs where ALS, post-cardiac arrest care, and cardiovascular interventions are available. Patients (>18 years old) whopresented with presumed OHCA and used the EMS system in Daegu were included in the study. Those who did not receive any resuscitationand those who experienced cardiac arrest in a primary care clinic or long-term care hospital were excluded. From 18 February 2020 to 31 March 2020 (the peak of the COVID-19 outbreak), 184 adults developed OHCA.Of them, 170patients were included in this study (Fig. 1). The study was approved by the Institutional Review Board ofKyungpook National UniversityHospital.(KNUH 2020-04-032)

Study design and variables
We conducted a multicenter observational study using the WinCOVID-19 citywide OHCAregistry dataand evaluated patient demographics (age andsex), prehospital variables (place of arrest, initial cardiac rhythm,whether it was witnessed by a bystander or EMS personnel,whether a bystander performed CPR, whether an automated external de brillator (AED) was used and prehospital de brillation occurred, and whether ROSC occurred), and time intervals (response time was de ned as the period from contactingthe EMS to themarrivingat the scene,while scene time was de ned as the period fromthe EMS arrivingat the scene to themdeparting from the scene). Resuscitation management and survival outcomes (including ROSC, survival to admission, survival to discharge,and neurological outcomes at discharge) aftercardiac arrest were recorded as hospital-level variables. Based on this collective data, we determined whether the patients met the existing TOR guidelines and quali ed to be enrolled in this study. In South Korea, EMS personnel cannot terminate resuscitation unless OHCA patients show obvious signs of death [7,11];therefore, all EMS-treated OHCA patients should be transported to the hospitals.Notably, these personnel use manual de brillators and analyze electrocardiograms(ECGs) prior tothe patients' arrival at the hospital.
Toevaluate COVID-related variables, we obtained data, including the clinical symptoms or signs, vital signs (fever[de ned as a temperature of 37.5°C or higher], sore throat, and cough), recent exposure history, and chest X-ray ndings, of patients with con rmed COVID-19 from electronic medical records on admission.

Main outcome measures
Primary outcome measures were based on the proportions of survival to hospital discharge (de ned as discharged to go home or transferred to another facility after admission to the hospital) and survival with a favorable neurological outcome (quanti ed using Cerebral Performance Category scores, with scores of 1 and 2 being the most favorable) in the group.
Secondary outcome measures included external validation of previously researched TOR rules (Table 1). We delineated OHCA patients who met all the TOR criteria and calculated sensitivity, speci city, false positive ratios (FPR), false negative ratios, positive predictive values (PPV), negative predictive values (NPV), and their respective 95% con dence intervals (CI) to identify patients with a risk of poor survival [7,8,11,13,14]. Logistic regressions were performed for patients who met the criteria of the previous TOR rules. The outcome of interest was survival to hospital discharge (Table 1). To determine the model calibration, the Hosmer-Lemeshow goodness of ttest was calculated. Odds ratio (OR) greater than 1 indicatedan unfavorable effect on survival; ORs and 95% CIs were calculated for all covariates. The characteristics of the adjusted ORswere described using Forest plots. A receiver operating characteristic (ROC) curve, with the area under the curve (AUC), was used to determine the accuracy of such variables in predicting unfavorable survival outcomes at discharge. All tests were two-tailed, and p<0.05 was considered statistically signi cant.
Diagnostic performances of allexistingTOR rules were calculated. Previous TOR rules were created based on FPR (the probability that the rule will suggest terminating resuscitation in case of patient survival) and PPV (the probability that the rule will suggest terminating resuscitation in case of patient death) of the rules. Current guidelines recommend that when cessation of life-sustaining care is considered, the tool used to predict poor outcomes must be accurate and reliable with an FPR (close to 0%) with a narrow 95% CI (0%-10%). PPV must be greater than 99% [7,15].Imprecision was graded as 'serious' when the upper limit of the 95% CI for FPR was greater than 5% and 'very serious' when thesame value was greater than 10%. Medical futility and drug therapy were found to impart a less than 1% chance of survival.

Patient characteristics
Of the 170 patients, 63.2% were men, 76.0% was witnessed by EMS personnel, 34.3% received CPRfrom a bystander, 4.7% experienced ROSC prior to hospital arrival, 58.5% showed abnormal chest X-rays,and 5.8% were con rmed to have COVID-19 by reverse transcription polymerase chain reaction(RT-PCR). All baseline characteristics of the study population are summarized in Table 2. Factors associated with survival The rate of survival to discharge was 4.7%, and that of survival with favorable neurological outcomes was 2.9%. In general,survivors, compared to the patients who died, were younger, experienced more prehospital de brillations, and achieved ROSC before arriving at theemergency department (ED) ( Table   2). Advanced age, not being witnessed by EMT personnel, receiving no shocks,and not developing ROSC before arrivingat the ED were factors related to unfavorable survival outcomes. Only ROSC before arriving at the ED was related to survival to discharge (Fig. 2). Interestingly, age, witness status, and prehospital shock delivery were signi cantly associated with survival outcomes in the univariate analysis, but not in the multivariable analysis (Table 3).

External validation of the TOR rules
On applying the existing internationalTOR rules to our registry, we found that 122 patients (71.8%) met all three criteria,but one survivor was discharged (Fig. 1).Another patient met the criteria for the traditional BLS-TOR andKoCARC TOR II rules, buthetoo was discharged( Table 4). The ROC curves of multimodal TOR categories for poor outcomesare shown in Figure 2. Overall, KoCARC TOR rule I and the currentTOR rule II, with AUCs of 0.951 and 0.967, respectively,could be used most effectivelyfor predicting in-hospital mortality (Fig. 3). In terms of the criteria of FPR (upper limit of 95% CI <5%) and PPV (>99%), only KoCARC TOR rule I could be used to predict poor survival outcomes, and it showed a higher diagnostic performance than the other TOR rules (Table 4).

Discussion
In this study, we validatedthe existingmultimodal TOR rules using the WinCOVID-19 Daegu registry data on OHCA patients.This was the rst study to determine the e cacy of these TOR guidelinesfor an emerging infectious disease.Among the nine existing TOR rules, KoCARC TOR rule I, having the lowest FPRand highest PPV, was found to be the best indicator of poor outcomes during the COVID-19 outbreak.

Resuscitation of infected individuals greatly increases the risk of transmission of virus to healthcare
providers [1,6,16]. According to the guidelines, emergency personnel are recommended to con rm the presence of COVID-19 in OHCA patients and wear high-level PPE and perform CPR. They must wear PPE, even when in doubt. However, this new type of corona infection has a very high rate of disease transmission and is characterized by approximately 25% of asymptomatic infection [4]; therefore, the COVID-19 era was a confusing time for healthcare workers in the many medical and emergency elds [5].
Some OHCA patients were unexpectedly con rmed to be positive post-CPR or on performing postmortem,contributing to emergency room shutdown and temporary closure [17].As of April 2020, more than 10,000 people have been con rmed to have COVID-19 and 167 COVID-19 patients have died in South Korea. More than 70% of these cases occurred in Daegu, the area of interestin this study, and Gyeongbuk, the neighboring area [10].More than 130 cases of infected healthcare workers and medical staff have been reported.One physician died while treating two con rmed patients.
Our ndings, although preliminary, showed that survival outcomes of OHCA patients in Daegu during the peak of the COVID-19 outbreak (4.1%) were signi cantly lower than those reported nationwide (9.8%) or in the city (8.8%) in 2018 [18]. We could not describe the broken chain of survivaland the negative effects on high-quality CPR in the results of the present study;however, due to this pandemic, the risk-bene t balance for CPRshould be reconsidered [2].Other studies have also raised the question of how CPRmust be performed for IHCA patients with con rmed COVID-19.Considering the lower survival rate, physicians should be concerned about the goals of care or CPR preferences to reduce futile resuscitation by stratifying survivability of the IHCA patients, regardless of their COVID-19 status, at the time of hospital admission [5,19]. It was also important to consider prehospital TOR for out-of-hospital resuscitation in an infectious disease epidemic area.
The previously existing TOR rules can be divided into two combined sets of variables, one of which can be applied at the pre-hospital level and the other can be evaluated immediately after arriving at the ED. In this study, we selected and analyzed the external validation of all nine multimodal TOR rules for OHCA patients during the COVID-19 epidemic period. These rules were typically selected depending on the country or region where the derivation and validation phaseswere conducted.These included the (1) International BLS(acombination of three criteria, including not being witnessed by emergency medical technicians (EMT), not receiving prehospital shock delivery, and not experiencing prehospital ROSC, see Table 1) and ALS rules derived and validated in the United States and Europe [8], (2) Goto and KANTO-SOS rules developed in Japan and Asian countries [13,14], and (3) Korean OHCA registry-based TOR models, KoCARC TOR rules, and two new TOR rules, which were used in our previous studies [7,11]. The international BLS-TOR rules that can be enforced at the prehospital stage show high sensitivity and speci city, but also relatively high FPR (upper limit of 95% CI of FPR>5%) [9,15];therefore, continuous development of the TOR model has been proposed [7,9].The researchers also proposed a new TOR model 1 that was applicable at the prehospital stage and a new TOR model 2 that was applicable immediately after arriving at the ED in previous studies by including acquired ECG asystole rhythm as a criterion [11].
The previous four rules have been partially validated in other countries and in the setting of mechanical CPR or comprehensive post-resuscitationcare [12,[20][21][22][23]. Previous validations of the TOR rule reported survival rates of less than 1% among TOR rule-positive patients in North America; however, high FPR of survival has been reported in Asian countries (28.7% in Singapore, 25.9% in Taiwan, and 30.4% in South Korea). This discrepancy may be due to different prehospital practices and a relatively higher prevalence of non-shockable rhythm in patients in Asian countries [7]. However, high FPRs of survival in these Asian countries, where the withdrawal of life-sustaining treatment is not commonly applied, are likely to be biased. Kajino et al. [24]validated the TOR rules for predicting poor neurologic outcomes in a Japanese population and concluded that more speci c TOR rules for each region should be developed, despite the good performance of the TOR rules in their study. However, even if the COVID-19 outbreak was not taken into account, these previous results implied that the extrapolation to and implementation of different TOR rules in regions with different organization of EMS treatment protocols, legislation, and socioeconomic characteristics might be problematicbecause the TOR rules would need to be adjusted to meet the regional situation.
In this study, we validatedthe existing multimodal TOR rules using the WinCOVID-19 Daegu registry data on OHCA patients. Based on the results of our study, we failed to screen one survivor, out of the seven survival discharges out of 170 OHCA patients, for the international BLS and KoCARC II rules; however, the remaining seven TOR rules were classi ed correctly. Based on current guidelines, it is recommended that diagnostic tests that guidethe cessation of life-saving efforts be accurate and reliable and show an FPR value close to 0% [7,9].Among the nine rules,KoCARC TOR rule I,showing the lowest FPR (upper limit of 95% CI <5%) and highest PPV (>99%), was found to be the most effective indicator for poor outcomes.
This rule included the combinationof three factors, including not being witnessed by EMT, presenting with an asystole at the scene, and experiencing no prehospital shock or ROSC. It did not include the patient's age or ED parameters, thereby being easyto use in prehospital settings and applicable for OHCA patients in this current pandemic [7].

Limitations
This studyhad several strengths and limitations. First, as with other multicenter observational studies,this study showed that the integrity of the data could be biased. Additionally, our observation period lasted onlytwo months;hence, the effects on long-term survival outcomes, which are most important for OHCA research, remain unknown. In this regard, we will continue to conduct investigations until COVID-19 has been effectively tackled. Second, it has been speculated thatpoor survival outcomes areassociatedwithfewer resuscitationsand prolonged EMS scene and transport timeto the hospital.
Moreover, unfavorable neurological outcome, as a primary outcome,is more suitable than survival to discharge. Since only four patients in our study displayedfavorable neurological outcomes, we could not perform any secondary analyses or external validations. Third, the ratio of the survival outcomes was much lower in our study thanin previous Korean OHCA data reports developed during the COVID-19 outbreak, and it is possible that this was due tothe direct adverse effects of COVID-19 on the cardiovascular system, prolonged EMS scene time,and differences in treatment in different hospitals [25].
Finally, facility or regional differences in EMS resources, CPR quality, and post-cardiac arrest care might affect the survival outcomes during the COVID-19 outbreak. TOR rules in the COVID-19 era and socioethical issues must be discussed further, and a consensus process must be developed.

Conclusions
Among the nine previouslyexisting TOR rules, KoCARCTOR rule I was found to be the most suitable for predicting poor survival outcomes, and it showed improved diagnostic performance in the COVID-19 era. With regard tothe FPR and PPV criteria, KoCARCTOR rule I, which included a combination of three factors including not being witnessed by EMT, presenting with an asystole at the scene, andexperiencing no prehospital shock or ROSC,was superior toall other TOR rules.Further research on variations in resources andtreatment protocols (CPR quality and post-cardiac arrest care) among facilities, regions, andcultures will be useful in determining the feasibility of TOR rulesfor COVID-19 patients worldwide.

Availability of data and materials
The data that support the nings of this study are available from the corresponding author upon reasonable request.

Figure 2
Forest plot for predicting unfavorable survival outcomes ofCOVID-19 patients with out-of-hospital cardiac arrest