Inter-observer variation in the triage-process

Background

Triage algorithms for sorting patients according to their severity of illness have recently been implemented at several emergency departments in Denmark. Clinical effectiveness and safety depend on standardization of the triage process. We implemented a 5-level triage algorithm for which triage nurses underwent an initial training. The aim of this study was to evaluate the degree of standardization in implementation of triage by measuring the level of agreement among retrospective reviewers of the triage scores.

Methods

Six months after implementation, we studied the inter-observer variation among our 5 triage nurses. Cases triaged by each nurse underwent two separate retrospective evaluations by all 5 triage nurses based on documentation of a) pre-arrival information, and b) pre-arrival information plus triage vital signs and clinical information gathered on arrival. Kappa-statistic was used to evaluate pair-wise agreement among the retrospective reviewers for each of the two retrospective evaluations, and also between the original triage score and second retrospective review.

Results

A total of 100 cases (20 consecutive cases triaged by each nurse) were evaluated. The distribution of initial triage scores was 1 red (highest acuity), 27 orange, 48 yellow, 24 green, zero blue (lowest acuity). Weighted kappa-score for pair-wise agreement among triage nurses for the two retrospective reviews were 0.40 (range: 0.24-0.56), and 0.41 (range: 0.38-0.46). Weighted kappa-score for pair-wise agreement between original triage score and each retrospective reviewer (pre-arrival information + triage vital signs and clinical information) was 0.45 (range: 0.34-0.58).

Conclusion

These results show, for the first time in Denmark, that moderate agreement can be achieved within a short period of time following implementation of an emergency department triage program with a brief initial training. A limitation of this study is that we did not systematically evaluate the completeness of the clinical documentation upon which the retrospective triage evaluations were made. Variability in the completeness of the clinical documentation may contribute to variability in the scoring. Quality assurance feedback and continuing triage nurse education are indicated to improve the standardization of triage scoring.