Mental health diagnostic interviews are less consistent than thought

Diagnostic interviews are widely used by mental health professionals to identify conditions such as anxiety, bipolar disorder and depression in adults, but new research led by McMaster University shows that the long considered "gold standard" may not be as consistent as previously thought.

The meta-analysis, published in JAMA Network Open on May 28, found that standardized diagnostic interviews are only moderately consistent when the same person is assessed more than once. In some cases, individuals were identified as having different diagnoses when interviews were repeated just days apart. Further, this consistency varies considerably depending on which disorder is being assessed.

The study's conclusion is particularly concerning as inconsistent diagnoses can lead to over‑ or under‑treatment, delayed care, or inappropriate interventions, highlighting the need for improvements in diagnostic tools and for greater caution when relying on a single interview to define psychiatric disorders.

Our findings show that these interviews are not as reliable or consistent as many people believe. If we give the same interview to the same person twice, we would like to think the interview would produce the same result, but that's not always the case."

Laura Duncan, senior author, assistant professor with McMaster's Department of Psychiatry and Behavioural Neurosciences and researcher, Offord Centre for Child Studies

The study found diagnostic interviews were more consistent for substance use disorders than for many mental disorders. Researchers say this may be because conditions such as anxiety, depression or psychotic disorders are more subjective experiences and for this reason, standardized interviews should be used with recognition of their limitations. .

"These differences suggest that structured interviews work better for conditions with clearer behaviours or timelines than for disorders that rely heavily on personal experiences and interpretation," Duncan says. "But we should reconsider treating them as a "gold standard" of assessment. Reliable diagnosis likely requires combining standardized tools with knowledge about the course and complexity of disorders that could impact how reliably they can be assessed. ."

Standardized diagnostic interviews are structured assessment tools commonly used to diagnose mental health disorders. As part of the interview, a person is asked about their mood, behaviour changes, along with questions about frequency and severity of symptoms and sometimes how much their symptoms impact their day-to-day life.

To conduct the study, researchers meta-analyzed 57 studies involving more than 8,000 adults from 26 countries. The research focused on test–retest reliability, a measure of whether an interview gives the same result when repeated with the same person and same interview under similar conditions, usually after 7-14 days.

The study was supported by collaborators with the University Hospital Copenhagen, UMass Chan Medical School, and St. Joseph's Healthcare Hamilton.