An effective test (EOmciSS) for screening older adults with mild cognitive impairment in a community setting : development and validation study

Abstract

Background: Early detection of mild cognitive impairment (MCI) symptoms is an important step to its diagnosis and intervention. We developed a new screening test called “Efficient Online MCI Screening System” (EOmciSS) for use in community-dwelling older adults. It is a self-paced cognitive test to be completed within 10 minutes on tablets or smartphones in homes or care centers for older adults. Objective: This study aims to test the validity of EOmciSS for identifying community-dwelling older adults with MCI risks. Methods: Participants (N=827) completed EOmciSS and other screening tests for MCI. The psychometric properties tested were “subscale item difficulty,” “discriminative index,” “internal consistency,” and “construct validity.” We also tested between-group discrimination using the cross-validation method in an MCI group and a normal cognitive function (NCF) group. Results: A total of 3 accuracy factors and 1 reaction time factor explained the structure of the 20 item factors. The difficulty level of accuracy factors (ie, “trail making,” “clock drawing,” “cube copying,” “delayed recall”) was 0.63-0.99, whereas that of the reaction time factor was 0.77-0.95. The discriminative index of the medium-to-high-difficulty item factors was 0.39-0.97. The internal consistency (Cronbach α) ranged from .41 (for few item factors) to .96. The training data set contained 9 item factors (CC-Acc1, P<.001; CD-Acc1, P=.07; CD-Acc2, P=.06; CD-Acc3, P<.001; TM-Acc4, P=.07; DR-Acc1, P=.03; RS, P=.06; DR-RT1, P=.02; and DR-RT2, P=.05) that were significant predictors for an MCI classification versus NCF classification. Depressive symptoms were identified as significant factors (P<.001) influencing the performance of participants, and were an integral part of our test system. Age (P=.15), number of years of education (P=.18), and proficiency in using an electronic device (P=.39) did not significantly influence the scores nor classification of participants. Application of the MCI/NCF cutoff score (7.90 out of 9.67) to the validation data set yielded an area under the curve of 0.912 (P<.001; 95% CI 0.868-0.955). The sensitivity was 84.9%, specificity was 85.1%, and the Youden index was 0.70. Conclusions: EOmciSS was valid and reliable for identifying older adults with significant risks of MCI. Our results indicate that EOmciSS has higher sensitivity and specificity than those of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment and the Computerized Cognitive Screen. The user interface, online operation, and self-paced format allowed the test system to be operated by older adults or their caregivers in different settings (eg, home or care centers for older adults). Depressive symptoms should be an integral part in future MCI screening systems because they influence the test performance and, hence, MCI risk.