Two observational studies were performed at the PED, Department of Pediatrics, St. Olav’s University Hospital, Trondheim, Norway, during the period from April 2013 to February 2014. The Department of Pediatrics provides emergency care for a population of approximately 58,000 children aged < 16 years, of whom 18,000 are less than 5 years of age (Statistics Norway, 2014). In 2013 the PED received 4223 children aged 0–16 with various pediatric (n = 3167, 75 %), surgical (n = 506, 12 %), neurosurgical (n = 211, 5 %), orthopedic (n = 106, 2.5 %), and other complaints (n = 233, 5.5 %). The largest patient group was children with respiratory tract infections (n = 699, 17 %). Children with life-threatening illness, including those with multitrauma, usually were not received in the PED but in the ED for adults and are not included in the studies. Nearly all children were referred after being assessed in the primary care system by general practitioners or at the primary care ED. In the PED more than half of the patients were treated as outpatients, and approximately 40 % were admitted to the pediatric wards. In 2013, RETTS-p triage priority ratings in the entire PED population were: red 8.7 % of the admitted children, orange 30.3 %, yellow 29.5 %, green 18.8 % and blue 12.7 %.
Description of the rapid emergency triage and treatment system for children
Triage with RETTS-p is based on a combination of VP measurements and evaluation of individual disease manifestations (Emergency Signs and Symptoms, ESS) (Additional files 1 and 2) [8, 11]. The VPs include airway, respiratory rate, oxygen saturation, heart rate, alertness level as measured by the Glasgow Coma Scale, and temperature, and the VP priority levels are age adjusted. Forty ESS algorithms each cover one or more of more than 100 common acute pediatric, surgical, and orthopedic complaints, e.g., respiratory difficulty and apnea, which are included in ESS no. 104, abdominal pain, constipation, and diarrhea (ESS no. 106), head trauma (ESS no. 130), and uncomfortable parents (ESS no. 153), respectively. Both VPs and ESSs are scored in one of five priority levels by ED nurses. Priority level red is defined as urgent need of medical doctor examination, orange implies medical doctor examination no later than 20 min, triage priority rating yellow means that the patient should be evaluated within no more than 120 min, and green demands examination before 4 h. Blue level indicates no need for triage and examination at the ED, and this patient group was not included in the present studies. The final triage priority rating is determined as the highest level from the VP and ESS ratings. Each ESS algorithm in addition includes recommendations for initial basic evaluation and treatment, such as blood sugar level testing and oxygen treatment. In the present studies, we used a Norwegian version of the RETTS-p (version 1.2), which was translated and adapted to Norwegian conditions by the research group.
Study design
Study 1
Prior to the introduction of RETTS-p in 2012, all nurses in the PED completed a theoretical and practical training. Four months later, 19 nurses triaged 20 written but realistic patient cases (Additional file 1), which were based on representative real-life referrals to our PED (Wave A). Nine months later, 12 nurses who were still working in the PED triaged the same cases again (Wave B).
Study 2
Pairs comprising one out of 20 regular nurses and one out of four research nurses simultaneously and independently triaged 200 children who were referred with either pediatric (n = 150), surgical (n = 30), neurosurgical (n = 10) or orthopedic (n = 10) complaints during the time period from June 2013 to February 2014. The children were included when one of the research nurses was available on dayshifts and afternoon shifts. The distribution of the included children was quite similar to the distribution of children in the entire PED population: pediatric patients 75 vs 75 %, surgical patients 15 vs 12 %, neurosurgical patients 5 vs 5 % and orthopedic patients 5 vs 3 %. In the emergency room, the regular nurse triaged the patients as usual with written documentation of VPs, ESS number, and triage priority scores. Simultaneously, the research nurse as a silent observer received all objective measurements, apart from evaluation of alertness and respiratory rate, which were observed by the research nurse herself, and independently triaged the patient. Usually, the research nurses worked in the PED and had more experience in pediatric nursing care compared with the regular nurses (mean 17 vs. 5 years). Also, the proportions with continuing education in nursing were higher (100 vs. 40 %, respectively).
Ethics
The study was approved by the institutional review board at St. Olav’s University Hospital and the Regional Ethics Committee, Mid-Norway. All nurses in our emergency department were asked to participate (orally). Written consent to participation in Study 1 was documented. In Study 2 the research nurse prior to triaging informed the regular nurse, the child and the caregiver, and oral consent to participation was collected from the regular nurses. The hospital review board approved this approach.
Statistical analysis
In Study 1, the inter-rater agreement was measured in Waves A and B separately using Kendall’s W coefficient of concordance [15], which has a similar interpretation as a correlation coefficient. In Wave A, 13 single values were missing for six nurses. That is, 13 values were missing out of 19 × 20 = 380 values (3.4 %). In Wave B, 4 single values were missing for four nurses. That is, 4 values were missing out of 12 × 20 = 240 values (1.7 %). The missing values were singly imputed using the expectation-maximization algorithm, making it possible to use all 19 (12) nurses in the analysis. We used a linear mixed effects model to estimate the variance components due to variance between patient cases and variance between raters, as well as to study intrarater agreement (i.e., whether there was a “learning effect” from Wave A to Wave B). The triage priority scores made by the nurses were furthermore compared with consensus triage priority scores as determined by the research group.
In Study 2, data were missing on 20 patients: Triage priority ratings were not available for the regular nurse in 5 patients and for the research nurse for 15 patients. Hence, 180 patient cases were available for analysis, including 20 regular nurses and four research nurses. We used a linear mixed model with triage priority score (Red = 1, Orange = 2, Yellow = 3, Green = 4) as dependent variable, patient and nurse as crossed random factors, and a fixed effect of research nurse (vs. regular nurse). The interrater reliability measured as the intraclass correlation coefficient (ICC) in this model was estimated as [16].
$$ \widehat{ICC}=\frac{Variance(patients)}{Variance(patients)+ Variance(raters)+ Residual}. $$
This ICC is equivalent to Cohen’s quadratic weighted kappa [17], as also noted by van Veen et al. [7].
The ICC was calculated using Stata 11, and the other analyses were done in SPSS 22.
