Study Finds Misclassification of Dementia by Brief Cognitive Assessments

Janice Ranson, MSc, PhD Student, University of Exeter Medical School

Janice Ranson, MSc, PhD Student

A new study has found that brief cognitive assessment, particularly used in a primary care setting, can result in false-positive and false-negative misclassification of dementia, largely due to test specific biases.

Researchers sought to detect predictors of misclassification when utilizing 3 brief cognitive assessments to detect possibly dementia in primary care: the Mini-Mental State Examination (MMSE), Memory Impairment Screen (MIS) and animal naming (AN). Potential consequences of misclassification depend on the clinical circumstance and context. False-negatives tests could result in prevention or delay of diagnosis, while false-positive tests may cause unnecessary referrals affecting all parties involved.

“Dementia can be difficult to accurately detect, particularly in a primary care setting,” lead author Janice M. Ranson, MSc, University of Exeter Medical School told NeurologyLive. “We found that while these tests are often useful, they can result in false positive or false negative results, and we were really interested in what factors may be at play when the tests get it wrong. We found the tests have independent biases, so different patient groups are misclassified by different tests. This means that in practice, the accuracy of each test will vary with the characteristics of the patient such as education level, ethnicity and age.”

The analysis was based on data from 824 older adult participants in the population-based US Aging, Demographics and Memory Study with adjudicated dementia diagnosis (Diagnostic and Statistical Manual of Mental Disorders (DSM) III-R and DSM-IV-IV criteria) as the reference standard and had complete data on all outcome and predictor variables. Specific diagnoses were then grouped into 3 diagnostic categories: all-cause dementia; cognitive impairment, not dementia (CIND); and normal cognitive function.

The prevalence of dementia in the sample of participants was 35.3% (n=291) and of those without dementia 43.3% (n=231) met the criteria for CIND. The participants had a mean age of 81.62 years (standard deviation (SD)=7.11), 10.14 mean years of education (SD=4.29), 479 (58.1%) females and 77 (9.3%) residents in a nursing home. The ethnicity consisted of 595 (72.2%) Caucasians, 148 (18%) African Americans and 81 (9.8%) Hispanics.

The results revealed that misclassification by at least 1 assessment occurred in 301 (35.7%) causes, misclassification by 2 or more assessments occurred in 113 (13.4%), and 14 (1.7%) were misclassified by all 3. The overall dementia misclassification rates for the MMSE, MIS, and AN were 21%, 16%, and 14%, respectively. The number of participants with false-positives that met the criteria for CIND were 114 (74.5%), 64 (82.1%), and 46 (82.1%), respectively, while the number of true-negatives that met the criteria for CIND were lower—117 (30.8%), 167 (36.7%) and 185 (38.8%), respectively.

“Our results suggest that knowing the specific limitations of each test can help clinicians decide which test is the most appropriate for their patient,” Ranson explained. “There is currently no strong evidence to suggest one particular test is best for everyone. From our findings, it appears that currently the best test depends on the clinical context and patient characteristics. Our results also highlighted the importance of obtaining informant-reported memory ratings of the patient when possible, for example from a spouse or close family member. This can be taken into account when interpreting the brief test results.”

Researchers identified different patterns of predictors for misclassification by each assessment and found 7 variables that predicted false-negatives on only 1 assessment: For the MMSE, years of education predicted higher false negatives (odds ratio (OR) 1.23, 95% CI 1.07—1.40); age, APOE E4 non-carrier, depression, and absence of informant-rated memory decline for MIS; not residing in a nursing home (OR 4.85, 95% CI 1.27—18.45) and physical activity for AN. The absence of self- and informant-rated poor memory predicted false-negatives on the MIS and AN; and there were no consistent predictors of false-negatives on all assessments.

There were 5 variables that predicted false-positives on only 1 assessment: Lower education (OR 0.77, 95% CI 0.70—0.83), illiteracy, visual impairment, and APOE E3 for MMSE; and Hispanic ethnicity for AN. There were no predictors reported specific to false-positives on the MIS. Three predictors were associated with false-positives on 2 assessments: Age for the MIS and AN; and nursing home residency and African-American ethnicity for the MMSE and AN. Similarly, to the false-negative predictors, there were no consistent predictors of false-positive across all assessments. While a wide range of predictors were identified, there was only 1 predictor consistent across all assessments: Absence of informant-rated poor memory for overall misclassification.

“Brief tests like these are routinely used in primary care to help clinicians decide which patients would benefit from a referral for a full dementia investigation, which involves a longer, more comprehensive assessment,” said Ranson. “Brief tests are an important initial filter in the early stages of the dementia pathway, to identify those likely to have dementia. It is vital that we work to improve this initial stage of dementia identification, to avoid missed cases and unnecessary referrals.”

When asked if new tests should be created to avoid bias, Ranson explained that “Previously, there has been a tendency to simply create new tests or adjustments in an attempt to overcome limitations of existing tests. Clinicians now have many brief tests to choose from, which all have limitations and biases and there is a lack of evidence-based guidance for selecting the right one. This research has motivated us to think differently about cognitive testing. We are now developing new technology using machine learning and artificial intelligence to help clinicians get the best outcome for their patients."

“Looking forward, there is definitely a great deal of room for improvement in the diagnostic pathway, particularly this initial stage of brief testing,” Ranson concluded. “We desperately need more accurate and less biased ways of detecting dementia swiftly in clinic. We are currently working hard to improve these conventional tests with a more personalized and adaptive approach to dementia identification.”

REFERENCE

Ranson J, Kuźma E, Hamilton W, et al. Predictors of dementia misclassification when using brief cognitive assessments. Neurology. 2019;9(1):1—9. doi: 10.1212/CPJ.0000000000000566.