Skip to main content
Download PDF
  • Original research
  • Open access
  • Published:

Performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis screening tools with respect to identification of sepsis among ambulance patients with suspected infection: a prospective cohort study

Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine volume 29, Article number: 144 (2021) Cite this article

Abstract

Background

There is little evidence of which sepsis screening tool to use in the ambulance setting. The primary aim of the current study was to compare the performance of NEWS2 (National Early Warning score 2) and RETTS (Rapid Emergency Triage and Treatment System) with respect to identification of sepsis among ambulance patients with clinically suspected infection. The secondary aim was to compare the performance of the novel Predict Sepsis screening tools with that of NEWS2, RETTS and clinical judgment.

Methods

Prospective cohort study of 323 adult ambulance patients with clinically suspected infection, transported to hospitals in Stockholm, during 2017/2018. The sensitivity, specificity, and AUC (Area Under the receiver operating Curve) were calculated and compared by using McNemar´s test and DeLong’s test.

Results

The prevalence of sepsis in the current study population was 44.6% (144 of 323 patients). No significant difference in AUC was demonstrated between NEWS2 ≥ 5 and RETTS ≥ orange. NEWS2 ≥ 7 demonstrated a significantly greater AUC than RETTS red. The Predict Sepsis screening tools ≥ 2 demonstrated the highest sensitivity (range 0.87–0.91), along with RETTS ≥ orange (0.83), but the lowest specificity (range 0.39–0.49). The AUC of NEWS2 (0.73) and the Predict Sepsis screening tools (range 0.75–0.77) was similar.

Conclusions

The results indicate that NEWS2 could be the better alternative for sepsis identification in the ambulance, as compared to RETTS. The Predict Sepsis screening tools demonstrated a high sensitivity and AUCs similar to that of NEWS2. However, these results need to be interpreted with caution as the Predict Sepsis screening tools require external validation.

Trial registration: ClinicalTrials.gov, NCT03249597. Registered 15 August 2017—Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03249597.

Background

Sepsis is one of the most common medical emergencies and the mortality is high [1,2,3]. Sepsis is, however, often not identified in a timely fashion [4,5,6] despite the knowledge that time to treatment is related to patient outcome [7,8,9,10]. Time to treatment has been shown to be halved when sepsis is identified in the ambulance [11]. Hence, identification of patients likely to develop sepsis in this setting is important as more than half the patients with sepsis arrive to hospital by ambulance [12].

Screening tools have been shown to increase sepsis identification as compared to clinical judgment [5, 6], but there are a few screening tools developed explicitly for the identification of sepsis in the ambulance [13,14,15,16,17,18,19]. Neither the National Early Warning score (NEWS2) [20] nor the rapid emergency triage and treatment system (RETTS) [21, 22], an early warning score and a triage system respectively [20], are initially designed to identify sepsis. The use of NEWS2 is increasing worldwide [20]. It has been implemented in most hospital wards in addition to some emergency departments (EDs) and is gaining interest with some of the ambulance services in Sweden. RETTS is a triage system initially developed in Sweden [23] and is currently the most used triage system both in the ambulance and EDs. Both NEWS2 and RETTS have been proposed to be used to identify sepsis among patients with suspected infection [20, 24, 25], while NEWS2 has been shown to be superior to RETTS in the ED setting [22]. Neither NEWS2 nor RETTS have previously been validated with respect to sepsis identification in the ambulance.

Both NEWS2 and RETTS are based primarily on vital signs. However, more than one third of the patients with severe infection present with normal vital signs [26]. This suggests that including variables other than vital signs is needed for sepsis screening which was also the rationale for the development of the Predict Sepsis screening tools [27]. These tools are unique in that they were developed explicitly for sepsis identification in the ambulance and the result of a prospective, stepwise approach where the association with sepsis was calculated for each variable measured in the ambulance-also including symptoms.

The primary aim of the current study was to compare the performance of NEWS2 and RETTS with respect to identification of sepsis among ambulance patients with clinically suspected infection. The secondary aim was to compare the performance of the novel Predict Sepsis screening tools with that of NEWS2, RETTS and clinical judgment.

Methods

Study design and setting

The study was a prospective cohort study of 323 adult non-trauma, ambulance patients with clinically suspected infection transported to hospitals in Stockholm. We compared the performance of NEWS2 and RETTS for the identification of sepsis. Furthermore, the performance of the recently developed Predict Sepsis screening tools was compared with that of NEWS2, RETTS and clinical judgment. The current study was part of the Predict Sepsis study [27] (Clinical Trials identifier NCT03249597).

Selection of participants

Inclusion criteria were adult (≥ 18 years) non-trauma, ambulance patients, considered to suffer from a new onset infection according to clinical judgment by the ambulance personnel, and required data to determine the outcome sepsis/no sepsis. For details, see Predict Sepsis study [27].

All patients were enrolled by the ambulance personnel during the period of April 3rd, 2017 and August 30th, 2018 and transported by the ambulance provider Samariten Ambulans AB [28] to one of the seven major hospital EDs in Stockholm city county [27]. All ambulances were staffed with at least one nurse specialist and one emergency medical technician [29].

The exclusion criterium was participants lacking data required to complete each screening model.

Definition of outcomes

Sepsis

Sepsis was defined in accordance with the Sepsis-3 criteria [30], i.e., infection [6, 27] in combination with an increased SOFA score of ≥ 2 points, within 36 h from ED arrival [27]. The preexisting score was set to zero for patients with no previous recording of baseline data [27, 30]. Septic shock was defined as sepsis in combination with indication for vasopressor treatment and a serum lactate level greater than 2 mmol/L within 36 h from ED arrival [27, 30].

No sepsis

“No sepsis” was defined as not fulfilling above criteria for sepsis.

Sepsis screening models

NEWS2 (described in Table 1) is the 2017 updated version of NEWS, originally designed by the Royal College of Physicians in 2012 and it is based on six vital signs [20]. A NEWS2 score of 5 or more is used as indicative of potential serious acute clinical deterioration and the need for an urgent response [20]. A NEWS2 score of 7 or more is considered indicative of a severely ill patient, in need of an emergency response including personnel with critical care competence [20, 31].

Table 1 NEWS2, RETTS and the Predict Sepsis screening tools
Full size table

RETTS [21] is a triage system developed and licensed by Predicare AB [23]. It is a five-graded color scale, based on vital signs (VS, see Table 1 for a description) and Emergency Symptoms and Signs (ESS) which reflect presentation and symptoms. The most pronounced vital sign or ESS deviation will decide the triage level. Red is the highest triage level (defined as life threatening), followed by orange (potentially life threatening), yellow, green, and blue [23].

Sepsis, according to clinical judgment, was defined as the primary assessed condition sepsis (code C05) as recorded in the ambulance record.

The Predict Sepsis screening tools [27] are presented in Table 1. The Predict Sepsis screening tool 1 is based on symptoms, vital signs, and lactate. Predict Sepsis screening tool 2 is based on four variables of which two are vital signs and two are symptom-based. Predict Sepsis screening tool 3 is based on vital signs alone, but with novel cut-offs calculated to have the strongest association with the outcome sepsis.

Measurements; data collection and handling

Eight keywords related to medical history (“fever or suspected fever”, “pain”, “acute altered mental status”, “weakness of the legs”, “breathing difficulties”, “loss of energy”, “gastrointestinal symptoms” and “risk factors for sepsis”) and six vital signs (respiratory rate, oxygen saturation, heart rate, systolic blood pressure, Glasgow coma scale; GCS and temperature) were collected through a Case Report Form (CRF) used in the ambulance as part of the Predict sepsis study [27]. Priority level according to RETTS, vital signs not recorded in the CRF and primary assessed condition were extracted from the ambulance records (amPHI® Prehospital ambulance record, Amphi Systems A/S, Aalborg, Denmark, through the hospital medical record TakeCare®, v. 18.3.10, CompuGroup Medical, Stockholm, Sweden) and the local digital IT-support for prehospital care in Stockholm; FRAPP® (Framtida IT-plattform för prehospital vård i Stockholms läns landsting).

Data related to ED arrival time, age, gender, criteria for suspicion of a new-onset infection included in the Sepsis-3 definition of sepsis, in-hospital vital signs/ laboratory tests/ mortality and discharge International Classification of Diseases (ICD) code were extracted from the hospital medical records [27].

Statistical analyses

Statistical analyses were performed using SPSS (Statistical Package for the Social Sciences) version 27.0 (SPSS Inc., Chicago, IL, USA), and Clinical Research Calculators; Calculator 1, Vassarstats.net [32].

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and likelihood ratio (positive and negative LR) of NEWS2, RETTS, clinical judgment and the three Predict Sepsis screening tools were calculated in relation to outcome sepsis and outcome septic shock by Vassarstat.net, Clinical Calculator 1 [32]. The area under the receiver operating curve (AUC) was calculated (using SPSS) for the models without cut-offs (based on sum of scores) and with specific cut-offs. The sensitivity and specificity of each model for the outcomes sepsis and septic shock were compared using McNemar´s test. The AUC for the outcomes sepsis and septic shock was compared using DeLong´s test. P-values ˂0.05 were considered statistically significant.

Ethical approval

The study was approved by the Stockholm Regional Ethical Review Board (reference number 2016/2001-31/2, 2018/2202 and 2020-03894). Written consent was obtained from all participants.

The current study complied with the Declaration of Helsinki [33] and the manuscript was drafted according to the Standards for the Reporting of Diagnostic accuracy studies (STARD) criteria [34].

Results

Characteristics

551 patients with clinically suspected infection were included in the Predict Sepsis study [27]. The 323 patients that had the data required to complete each screening model were included in the current study. Of these, 144 (44.6%) had sepsis.

For characteristics of the study participants, see Table 2. Fifteen out of 144 septic patients (10.4%) died during in-hospital stay. The highest in-hospital mortality was observed among patients with RETTS red (7/48 patients, 14.6%) or NEWS2 ≥ 7 (15/123 patients, 12.2%), see Additional file 1.

Table 2 Characteristics of the 323 ambulance patients with suspected infection and the 144 patients with sepsis
Full size table

Performance of the screening models

See Table 3 for the performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis tools with respect to sepsis identification, and Additional file 2–3 for McNemar´s test for comparison of sensitivity and specificity, Additional file 4–5 for DeLong’s test for comparison of AUC for the models with and without specific cut-offs and Figs. 1 and 2 for the Receiver Operating Characteristics; ROC curves.

Table 3 Performance of the screening models with respect to identification of sepsis
Full size table
Fig. 1
figure 1

ROC curves for models without cut-offs and sepsis*. ROC, Receiver Operating Characteristic Curve; NEWS2, National Early Warning Score 2; RETTS, Rapid Emergency Triage and Treatment System. *ROC curves based on sum of scores for NEWS2 and the Predict Sepsis screening tools, with respect to outcome sepsis

Full size image
Fig. 2
figure 2

ROC curves for models with specific cut-offs and sepsis*. ROC, Receiver Operating Characteristic Curve; NEWS2, National Early Warning Score 2; RETTS, Rapid Emergency Triage and Treatment System. *ROC curves for all the models, with specific cut-offs, with respect to outcome sepsis

Full size image

NEWS2 compared to RETTS

No significant difference in performance was demonstrated when NEWS2 ≥ 5 and RETTS ≥ orange were compared (Table 3 and Additional file 5).

NEWS2 ≥ 7 demonstrated a significantly higher AUC than RETTS red (P-value < 0.001, see Table 3 and Additional file 5).

Predict Sepsis screening tools compared to NEWS2, RETTS and clinical judgment with respect to outcome sepsis

The Predict Sepsis screening tools demonstrated a significantly higher sensitivity (ranging between 0.87 and 0.91) and lower specificity (ranging between 0.39 and 0.49) as compared to NEWS2 (≥ 5 and ≥ 7), RETTS red and clinical judgment (see Table 3, Additional file 2, 3).

The AUC (based on sum of scores) of the Predict Sepsis screening tools (tool 1: 0.77, tool 2: 0.75, tool 3: 0.77) was similar to that of NEWS2 (0.73) (see Table 3, Additional file 4).

RETTS red demonstrated a significantly lower sensitivity (0.23) than all the other screening models except for clinical judgment (Table 3 and Additional file 2).

The specificity of RETTS red (0.92) was higher than that of all the other models (Table 3, Additional file 3).

Additional findings; comparison of performance of the screening models with respect to identification of septic shock

See Additional file 6–12 for the performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis tools, with respect to identification of septic shock.

The Predict Sepsis screening tool based solely on vital signs (tool 3) and RETTS ≥ orange identified 17/17 patients (100.0%) that developed septic shock (Additional file 6). RETTS red identified a significantly lower proportion of patients (8/17 patients, 47.1%) that developed septic shock, as compared to all screening models except for clinical judgment that identified 13/17 patients, 76%, see Additional file 6–7.

Discussion

This is the first prospective study to compare the performance of NEWS2 and RETTS in the ambulance setting for the identification of sepsis. The results of the current study indicated no major difference with respect to sepsis identification when based on comparisons of the AUC of RETTS orange, NEWS2 (both NEWS2 ≥ 5 and NEWS2 ≥ 7) and the Predict Sepsis screening tools. However, RETTS red and clinical judgment demonstrated a significantly lower AUC as compared to the other models with respect to sepsis. The Predict Sepsis screening tools showed promising results of a high sensitivity but, conversely, a low specificity.

The performance of the screening models

A NEWS2 score of 5 or more identified three of four septic patients and nine of ten patients who developed septic shock. The Royal College of Physicians recommends a NEWS2 score of 5 or above to be considered as suspected sepsis among patients with clinical suspicion of infection and recommend a rapid escalation of clinical care in addition to urgent treatment for these patients [20]. There is an ongoing discussion [35] to apply a NEWS2 cut-off of 7 or higher to identify the sickest septic patients. This cut-off is supported by the results of the current study showing that eight of ten patients who developed septic shock were identified.

RETTS highest priority level (i.e., “red”) appears to be insufficient for sepsis identification due to the low sensitivity for sepsis. The low sensitivity may be explained by the cut-off for respiratory rate being high while that for GCS require an unconscious patient, resulting in a lower proportion of patients fulfilling these criteria. RETTS red has been suggested to be used to identify patients with severe sepsis and septic shock [36]. However, it failed to identify more than half of the patients who developed septic shock in the current study. A better alternative would be to use the second triage level, i.e., RETTS ≥ orange, which identified four of five septic patients and all patients who developed septic shock.

Four of ten patients that developed sepsis were identified by clinical judgment which was higher than previously demonstrated [5, 11]. Enhanced attention on sepsis, including clinical updates of the Swedish ambulance guidelines [37], may have contributed to these results. Additionally, sepsis awareness among ambulance personnel was likely to have been affected by the Predict Sepsis study itself. Nonetheless, the current results support that applying a screening tool increases sepsis identification.

The Predict Sepsis screening tools, of which the two first tools include symptom variables, demonstrated the highest sensitivity, together with RETTS orange, but a low specificity and the AUCs were similar to that of NEWS2. The major disadvantage of these tools was the low specificity. The Predict Sepsis tools did however capture almost all the patients who developed septic shock.

The choice of a screening tool; sensitivity versus specificity

It is a well described challenge to development of a screening tool combining both a high sensitivity and a high specificity. A low specificity may cause false sepsis alerts leading to an over-use of resources, while a low sensitivity may lead to missing septic patients resulting in an increased mortality and morbidity. We advocate that screening tools should have a high sensitivity and that false sepsis alerts could be reduced by the assessment of an experienced clinician after the initial screening, since the specificity of experienced clinicians has been shown to be high [6]. In our opinion, the screening model should be regarded as a first step in the clinical decision process that leads to a correct diagnosis.

The timing of the identification and treatment of septic patients without septic shock has been questioned [38, 39]. Nevertheless, we believe that all septic patients benefit from early identification as this not only allows for early treatment, but also enables monitoring of the patient from an early stage of care. Moreover, international guidelines, such as the Surviving Sepsis Campaign, recommend treatment within 1 h from the identification of all septic patients, not only for those suffering from septic shock [40].

Strengths and limitations of the current study

This is the first prospective study to compare the performance of NEWS2 and RETTS in the ambulance setting for the identification of sepsis, which is considered a strength of the study.

There are several limitations to the current study.

First, the Predict Sepsis screening tools were compared to NEWS2, RETTS and clinical judgment in the same population in which the Predict Sepsis tools were developed. This infers a risk of over-adapting the new model to the data material from where it was derived. Hence, the discriminative properties of the Predict Sepsis screening tools may be lower in another population and an external validation of the Predict Sepsis screening tools is therefore needed.

Second, calculation of the AUC based on sum of scores was not possible for RETTS since vital signs were registered but not the ESS data. However, all RETTS levels include information on ESS to decide the documented priority level and accordingly the calculated sensitivity and specificity are considered to be correct. Additionally, the AUC of RETTS with specific cut-offs was calculated and compared to that of the other models, in turn also given specific cut-offs.

Third, the results are based on the study population, i.e., patients with a suspected infection and are therefore not generalizable to the general ambulance population. Ideally, a sample representative of “all” ambulance patients should have been included for a screening tool to be applicable to the general ambulance population. This would have enabled study of the identification of patients that are not easily recognized as having an infection, e.g., the elderly with non-specific symptoms and those lacking fever. The inclusion of a sample of general ambulance patients was, however, not feasible at the time but would be of value in future studies.

Forth, the current study is the second part of the larger Predict Sepsis study [27]. The original power calculation was performed for the purpose of including enough patients with the outcome sepsis in relation to variables studied for the association with sepsis, and to develop the Predict Sepsis screening tools. Hence, the power calculation was not performed explicitly for the current study. However, we believe the results of comparing the performance of the screening models, also those in clinical use, in this study of prospectively included ambulance patients are of interest.

Finally, the Predict Sepsis study was not designed for the outcome septic shock and the number of patients who developed septic shock was small. The results relating to the performance of identifying septic shock should therefore be interpreted with caution and repeated in larger studies.

Conclusions

The results indicate that NEWS2 may be a better alternative than RETTS with respect to the identification of sepsis among patients with suspected infection in the ambulance setting. This conclusion is based on the results indicating that there is no difference between NEWS2 and RETTS when comparing the second highest priority levels, but a superior performance of NEWS2 when comparing the highest priority levels. The Predict Sepsis screening tools showed promising results with respect to a high sensitivity for sepsis and the AUCs were similar to that of NEWS2. However, these results need to be interpreted with caution as the Predict Sepsis screening tools require external validation.

Availability of data and materials

The data that support the findings of this study are available from Karolinska Institutet Södersjukhuset but restrictions apply to the availability of these data, which were used under license for the current study, and are not publicly available. Data are however available from the authors upon reasonable request and with permission of Karolinska Institutet Södersjukhuset.

Abbreviations

NEWS2:

National Early Warning Score 2

RETTS:

Rapid Emergency Triage and Treatment System

AUC:

Area Under the receiver operating Curve

NCT:

National Clinical Registration

ED:

Emergency Department

ESS:

Emergency Symptoms and Signs

GCS:

Glasgow Coma Scale

CRF:

Case Report Form

FRAPP:

Framtida IT-plattform för prehospital vård i Stockholms läns landsting (“future Information Technology platform for prehospital care in Stockholm City Council”—the name of the ambulance medical record system of Stockholm)”

ICD:

International Classification of Diseases

SPSS:

Statistical Package for the Social Sciences

PPV:

Positive Predictive Value

NPV:

Negative Predictive Value

LR:

Likelihood Ratio

STARD:

Standards for the reporting of diagnostic accuracy studies

ROC:

Receiver Operating Characteristic

IQR:

Interquartile range

References

  1. Mouncey PR, Osborn TM, Power GS, Harrison DA, Sadique MZ, Grieve RD, et al. Trial of early, goal-directed resuscitation for septic shock. N Engl J Med. 2015;372(14):1301–11.

    Article  CAS  Google Scholar 

  2. Yealy DM, Kellum JA, Huang DT, Barnato AE, Weissfeld LA, Pike F, et al. A randomized trial of protocol-based care for early septic shock. N Engl J Med. 2014;370(18):1683–93.

    Article  CAS  Google Scholar 

  3. Peake SL, Delaney A, Bailey M, Bellomo R, Cameron PA, Cooper DJ, et al. Goal-directed resuscitation for patients with early septic shock. N Engl J Med. 2014;371(16):1496–506.

    Article  CAS  Google Scholar 

  4. Henriksen DP, Laursen CB, Jensen TG, Hallas J, Pedersen C, Lassen AT. Incidence rate of community-acquired sepsis among hospitalized acute medical patients-a population-based survey. Crit Care Med. 2015;43(1):13–21.

    Article  Google Scholar 

  5. Wallgren UM, Castren M, Svensson AE, Kurland L. Identification of adult septic patients in the prehospital setting: a comparison of two screening tools and clinical judgment. Eur J Emerg Med Off J Eur Soc Emerg Med. 2014;21(4):260–5.

    Article  Google Scholar 

  6. Wallgren UM, Antonsson VE, Castren MK, Kurland L. Longer time to antibiotics and higher mortality among septic patients with non-specific presentations—a cross sectional study of Emergency Department patients indicating that a screening tool may improve identification. Scand J Trauma Resuscit Emerg Med. 2016;24(1):1.

    Article  Google Scholar 

  7. Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med. 2006;34(6):1589–96.

    Article  Google Scholar 

  8. Ferrer R, Artigas A, Suarez D, Palencia E, Levy MM, Arenzana A, et al. Effectiveness of treatments for severe sepsis: a prospective, multicenter, observational study. Am J Respir Crit Care Med. 2009;180(9):861–6.

    Article  CAS  Google Scholar 

  9. Gaieski DF, Mikkelsen ME, Band RA, Pines JM, Massone R, Furia FF, et al. Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department. Crit Care Med. 2010;38(4):1045–53.

    Article  Google Scholar 

  10. Ferrer R, Martin-Loeches I, Phillips G, Osborn TM, Townsend S, Dellinger RP, et al. Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program. Crit Care Med. 2014;42(8):1749–55.

    Article  CAS  Google Scholar 

  11. Studnek JR, Artho MR, Garner CL, Jr., Jones AE. The impact of emergency medical services on the ED care of severe sepsis. Am J Emerg Med. 2010.

  12. Wang HE, Weaver MD, Shapiro NI, Yealy DM. Opportunities for emergency medical services care of sepsis. Resuscitation. 2010;81(2):193–7.

    Article  Google Scholar 

  13. Robson W, Nutbeam T, Daniels R. Sepsis: a need for prehospital intervention? Emerg Med J. 2009;26(7):535–8.

    Article  CAS  Google Scholar 

  14. Ljungström L. En utmaning till infektionsläkare: Gör omhändertagandet av patienter med akut svår bakteriell infektion (SBI) lika bra som omhändertagandet av akuta koronara syndrom! Infektionsläkaren. 2005;4.

  15. Bayer O, Schwarzkopf D, Stumme C, Stacke A, Hartog CS, Hohenstein C, et al. An early warning scoring system to identify septic patients in the prehospital setting: the PRESEP score. Acad Emerg Med Off J Soc Acad Emerg Med. 2015;22(7):868–71.

    Article  Google Scholar 

  16. Polito CC, Isakov A, Yancey AH 2nd, Wilson DK, Anderson BA, Bloom I, et al. Prehospital recognition of severe sepsis: development and validation of a novel EMS screening tool. Am J Emerg Med. 2015;33(9):1119–25.

    Article  Google Scholar 

  17. Baez AA, Cochon L. Acute care diagnostics collaboration: assessment of a bayesian clinical decision model integrating the prehospital sepsis score and point-of-care lactate. Am J Emerg Med. 2016;34(2):193–6.

    Article  Google Scholar 

  18. Hunter CL, Silvestri S, Ralls G, Stone A, Walker A, Papa L. A prehospital screening tool utilizing end-tidal carbon dioxide predicts sepsis and severe sepsis. Am J Emerg Med. 2016;34(5):813–9.

    Article  Google Scholar 

  19. Johansson N, Spindler C, Valik J, Vicente V. Developing a decision support system for patients with severe infection conditions in pre-hospital care. Int J Infect Dis Off Publ Int Soc Infect Dis. 2018;72:40–8.

    Article  Google Scholar 

  20. Physicians RCo. National Early Warning Score (NEWS) 2-Standardising the assessment of acute-illness severity in the NHS, Updated report of a working party December 2017. 19 December 2017. www.rcplondon.ac.uk: Royal College of Physicians; 2017.

  21. Widgren BR, Jourak M. Medical emergency triage and treatment system (METTS): a new protocol in primary triage and secondary priority decision in emergency medicine. J Emerg Med. 2011;40(6):623–8.

    Article  Google Scholar 

  22. Mellhammar L, Linder A, Tverring J, Christensson B, Boyd JH, Åkesson P, et al. Scores for sepsis detection and risk stratification - construction of a novel score using a statistical approach and validation of RETTS. PLoS ONE. 2020;15(2):e0229210.

    Article  CAS  Google Scholar 

  23. Magnusson CA-O, Herlitz J, Karlsson T, Jiménez-Herrera M, Axelsson C, Magnusson CAO, et al. The performance of the EMS triage (RETTS-p) and the agreement between the field assessment and final hospital diagnosis: a prospective observational study among children < 16 years. LID – 500 Initial assessment, level of care and outcome among children who were seen by emergency medical services: a prospective observational study. LID. 2019;88:1471–2431.

    Google Scholar 

  24. Rosenqvist M, Bengtsson-Toni M, Tham J, Lanbeck P, Melander O, Åkesson P. Improved outcomes after regional implementation of sepsis alert: a novel triage model. Crit Care Med. 2020;48(4):484–90.

    Article  Google Scholar 

  25. Mellhammar L, Linder A, Tverring J, Christensson B, Boyd JH, Sendi P, et al. NEWS2 is superior to qSOFA in detecting sepsis with organ dysfunction in the Emergency Department. J Clin Med. 2019;8(8):1128.

    Article  CAS  Google Scholar 

  26. Suffoletto B, Frisch A, Prabhu A, Kristan J, Guyette FX, Callaway CW. Prediction of serious infection during prehospital emergency care. Prehosp Emerg Care Off J Natl Assoc EMS Phys Natl Assoc State EMS Directors. 2011;15(3):325–30.

    Google Scholar 

  27. Wallgren UM, Sjölin J, Järnbert-Pettersson H, Kurland L. The predictive value of variables measurable in the ambulance and the development of the Predict Sepsis screening tools: a prospective cohort study. Scand J Trauma Resuscit Emerg Med. 2020;28(1):59.

    Article  Google Scholar 

  28. Ulf Kanfjäll CaSAA. Annual ambulance assignments Samariten Ambulans AB Stockholm. Samariten Ambulans AB; 2019.

  29. Årsrapport 2017 Prehospitala verksamheter i SLL Sect. 4.1 and 6.2 (2017).

  30. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.

    Article  CAS  Google Scholar 

  31. Pullyblank A, Tavaré A, Little H, Redfern E, le Roux H, Inada-Kim M, et al. Implementation of the National Early Warning Score in patients with suspicion of sepsis: evaluation of a system-wide quality improvement project. Br J Gen Pract. 2020;70(695):e381–8.

    Article  Google Scholar 

  32. Richard Lowry P. VassarStats; Clinical Research Calculators, Calculator 1. VassarStats: Website for Statistical Computation2019 [http://www.vassarstats.net/]. Available from: www.vassarstats.net.

  33. World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4.

    Article  Google Scholar 

  34. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Clin Chem. 2015;61(12):1446–52.

    Article  CAS  Google Scholar 

  35. Wattanasit P, Khwannimit B. Comparison the accuracy of early warning scores with qSOFA and SIRS for predicting sepsis in the emergency department. Am J Emerg Med. 2020.

  36. Rosenqvist M, Fagerstrand E, Lanbeck P, Melander O, Akesson P. Sepsis alert—a triage model that reduces time to antibiotics and length of hospital stay. Infect Dis (London, England). 2017;49(7):507–13.

    Google Scholar 

  37. Medicinska behandlingsriktlinjer för ambulanssjukvården 2020. Vårdgivarguiden: Falck, Samariten Ambulans AB, AISAB, Region Stockholm; 2020.

  38. Sterling SA, Miller WR, Pryor J, Puskarich MA, Jones AE. The impact of timing of antibiotics on outcomes in severe sepsis and septic shock: a systematic review and meta-analysis. Crit Care Med. 2015;43(9):1907–15.

    Article  CAS  Google Scholar 

  39. Singer M. Antibiotics for sepsis: does each hour really count, or is it incestuous amplification? Am J Respir Crit Care Med. 2017;196(7):800–2.

    Article  Google Scholar 

  40. Levy MM, Evans LE, Rhodes A. The surviving sepsis campaign bundle: 2018 update. Crit Care Med. 2018;46(6):997–1000.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge first and foremost all the patients willing to participate, Samariten Ambulans AB Stockholm’s ambulance personnel, including Torkel and Ulf Kanfjäll for including patients and for your understanding of the underlying incentive of the study, Veronica Vicente and Sven-Erik Norgren for the help with RETTS data and Laerdal, FALCK Foundation, the Emergency Department of Södersjukhuset and Örebro University for funding.

Funding

Open access funding provided by Örebro University. This study was supported by grants from Laerdal, Falck Foundation, the Emergency Department of Södersjukhuset, Stockholm, and Örebro University. The funders did not influence the interpretation of results, nor in writing the manuscript.

Author information

Authors and Affiliations

  1. Department of Clinical Science and Education, Karolinska Institutet, Södersjukhuset, Sjukhusbacken 10, 118 83, Stockholm, Sweden

    Ulrika M. Wallgren, Hans Järnbert-Pettersson & Lisa Kurland

  2. Fisksätra Vårdcentral (Primary Health Care Center), Fisksätra torg 20, 133 41, Saltsjöbaden, Sweden

    Ulrika M. Wallgren

  3. Department of Medical Sciences, Akademiska Sjukhuset, Uppsala University, 751 85, Uppsala, Sweden

    Jan Sjölin

  4. Department of Medical Sciences, Örebro University, Campus USÖ, Södra Grev Rosengatan 32, 701 12, Örebro, Sweden

    Ulrika M. Wallgren & Lisa Kurland

Authors
  1. Ulrika M. Wallgren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Sjölin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hans Järnbert-Pettersson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lisa Kurland
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

UW is responsible for the literature search, study concept and design, data acquisition, analysis and interpretation of the data and drafting the manuscript. JS has contributed to the study concept and design, the drafting of the manuscript and to a critical revision of the manuscript for important scientific intellectual content. HP participated in the statistical analyses, had the principal responsibility for critical revision of these and critically revised and contributed to the manuscript. LK has participated in the study conception and design, the interpretation of the data and results, the drafting of the manuscript and critically revising the manuscript for important scientific intellectual content. In addition, LK contributed with administrative and material support as well as supervision of the study. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lisa Kurland.

Ethics declarations

Ethics approval and consent to participate

The study received approval from the Stockholm Regional Ethical Review Board (reference number 2016/2001-31/2, 2018/2202 and 2020-03894). Written consent was obtained from all participants.

Consent for publication

Consent for publication of the Predict Sepsis screening tools (Additional file 1) was obtained from the original publisher BMC.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Observed in-hospital mortality for patients identified as septic by the models.

Additional file 2

. McNemar's test, pairwise comparison of sensitivity for sepsis.

Additional file 3

. McNemar's test, pairwise comparison of specificity for sepsis.

Additional file 4

. DeLong's test, pairwise comparison of AUC for sepsis for models without cut-offs.

Additional file 5

. DeLong's test, pairwise comparison of AUC for sepsis for models with specific cut-offs.

Additional file 6

. Performance of the screening models with respect to identification of septic shock.

Additional file 7

. McNemar's test, pairwise comparison of sensitivity for septic shock.

Additional file 8

. McNemar's test, pairwise comparison of specificity for septic shock.

Additional file 9

. DeLong's test, pairwise comparison of AUC for septic shock for models without cut-offs.

Additional file 10

. DeLong's test, pairwise comparison of AUC for septic shock for models with specific cut-offs.

Additional file 11

. ROC curves for models without cut-offs and septic shock.

Additional file 12

. ROC curves for models with specific cut-offs and septic shock.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wallgren, U.M., Sjölin, J., Järnbert-Pettersson, H. et al. Performance of NEWS2, RETTS, clinical judgment and the Predict Sepsis screening tools with respect to identification of sepsis among ambulance patients with suspected infection: a prospective cohort study. Scand J Trauma Resusc Emerg Med 29, 144 (2021). https://doi.org/10.1186/s13049-021-00958-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13049-021-00958-3

Keywords

Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine

ISSN: 1757-7241