We found a very high inter- and intrarater reliability of the Scandinavian triage system for children RETTS-p; i.e., there was a high degree of agreement between nurses when they evaluated how fast referred children should be seen by a medical doctor. The nurses rated 85 % written, realistic pediatric cases at the correct priority level, and nearly all (99 %) were rated correctly or within one adjacent level. Undertriage one level appeared in only 3.3 % of cases and three times less frequently than overtriage, which is important from the perspective of identifying urgent patients. The findings from the theoretical study with case scenarios were largely confirmed in real-life triaging of children who were referred with common pediatric and surgical complaints. Our findings are in line with data from a recent Swedish study where nurses in a mixed adult-pediatric ED in a general hospital triaged 40 fictive pediatric cases, but it is reassuring that we now have confirmed the repeatability of RETTS-p in real-life triage in a PED.

Our PED population includes relatively few children with life-threatening illness, and because in Norway nearly all cases are referred after evaluation in the primary care system, we did not receive many with mild diseases. These patient groups may be the most easy to triage. Therefore, it is noteworthy that in the two present studies, the variances in triage priority ratings between patients were larger than the variances between raters (nurses). Some patients just seem to be more difficult to triage than others, independent of the nurse’s experience. On the other hand, in the fictive case study, some nurses triaged more precisely than others, and the real-life data showed that the research nurses, who were more experienced, had a tendency to triage many patients at a lower level. Hence, less experience may explain at least some of the variation in rating between nurses.

All PED nurses were trained practically and theoretically in RETTS-p prior to the implementation of the system, and in Study 2 new nurses who started during the study period received the same training. In both studies we found a very high interrater agreement, and through testing two times in the fictive case study, we also found a high intrarater agreement but no learning effect during a 9-month period when the nurses gained more experience with the system. From these findings, we suggest that RETTS-p may be a credible and robust triage system that nurses quickly learn to manage.

Currently, no data support using one pediatric triage system instead of another [5], but test performances in the present studies were not lower than in studies of other pediatric triage systems [5]. RETTS-p was developed for Scandinavian countries, but available English versions will enable use abroad. Triage in RETTS-p uses both VP measurements and clinical signs and symptoms to prioritize children. In the present studies, we found that the final triage priority rating was based on similar VP and ESS priority levels in approximately half of the cases, whereas in the rest, the final triage priority rating was due to either the priority levels of the VP or the ESS, which may support the superiority of including both physiological measurements as well as clinical manifestations in the assessments. However, urgency in the need of help does not always equals disease severity, and therefore the evaluation of RETTS-p should be complemented with studies on the validity; i.e., it should be studied if triage priority ratings in RETTS-p are associated with markers of disease severity in PEDs.

There may be some limitations of the study. Because in Norway most children are referred to hospital PEDs after initial evaluation in the primary care system, the study populations were selected, including a few with mild conditions. In addition, only a few with life-threatening conditions were included, because they usually are handled at our adult ED. Both patient groups are likely to be the most easy to triage, leaving in our study populations an overweight of those who may be among the most difficult to triage. The studies were performed at a single center in a relatively large PED at a university hospital in Norway, and clinical data of included patients were not compared with the entire PED population. All these factors may limit the generalizability of our findings. In Study 1 it may be a limitation that only 12 out of 19 nurses were included in Wave B and the evaluation of intrarater agreement. However, from our experience, we believe there may be a substantial turnover of nurses in hospitals (e.g., in relation to pregnancy and continuing education). Because in Study 2 we included those nurses who were available during a 9-month period, it may be claimed that too many nurses were inexperienced with the RETTS-p system, despite the fact that all received standardized training. On the other hand, one may argue that both conditions may rather be advantages than limitations, because RETTS-p was studied under realistic, everyday circumstances in the PED.

The Scandinavian triage system RETTS-p for children was easily implemented in a large Norwegian PED, and we found a very high degree of agreement between nurses who used RETTS-p to prioritize children in patient case scenarios as well as in real-life triaging. RETTS-p may be a credible and robust triage system in PEDs, but it has not been validated yet.