Inter-rater and intra-rater reliability of the Chinese version of the short orientation-memory-concentration test in people with stroke

Abstract

Purpose: This study aimed to use three statistical methods to investigate the inter-rater and intra-rater reliability of the Chinese version of the Short Orientation-Memory-Concentration Test (C-SOMC) for people who have had a stroke. Methods: Forty-four participants (31 men and 13 women) with a mean age of 59.05 ± 10.79 years who have had experienced a single episode of stroke were enrolled in this study. To determine the inter-rater reliability of the C-SOMC, the test was administered to each participant on the same day by two raters (A and B) who each had more than 1 year of work experience. To determine intra-rater reliability, rater B re-evaluated 36 of the 44 participants with the C-SOMC on the subsequent day. Intraclass correlation coefficients (ICCs), paired-samples t-tests, and Bland-Altman plots were used to analyze the inter-rater and intra-rater reliability. Results: The evaluation of inter-rater reliability for the total score and item 1, 4, 5, 6, and 3/7 showed respective ICCs of 0.959, 0.918, 1.000, 0.942, 0.905 and 0.913, indicating excellent inter-rater reliability for the C-SOMC. Item 2 had an ICC of 0.796, indicating moderate to good inter-rater reliability. The evaluation of intra-rater reliability showed an ICC of 0.978 for the total score, and respective ICCs of 1.000, 1.000, 1.000, 0.968, 0.973 and 0.929 for the individual items, indicating excellent intra-rater reliability for the C-SOMC. The paired-samples t-test for the C-SOMC showed no statistically significant differences (P > 0.05) between ratings made by two different raters or by the same rater on separate occasions. The minimal detectable change value at the 95% confidence threshold (MDC95) of the SOMC score was found to be 2.14. Bland-Altman plots showed a mean difference of 0.02 and 95% limits of agreement (95% LOA) ranging from −3.86 to 3.90 for the inter-rater measurements and a mean difference of 0.33 and 95% LOA of −2.71 to 3.37 for the intra-rater measurements. Conclusions: The C-SOMC demonstrated excellent inter-rater and intra-rater reliability when administered to people who have had a stroke. The C-SOMC may be used to screen for cognitive impairment in people who have had a stroke.