What is a mental "disorder" (part 3): validity of diagnoses according to the Robins & Guze criteria (1970)

In psychiatry, validity has mostly been an issue when determining how “validly” a diagnostic instrument may identify a categorical diagnosis. Comparatively less attention has been focused on the validity of the diagnostic constructs. 

A few leading psychiatrists have developed criteria for the validity of disorders (Andreasen, 1995; Kendell & Jablensky, 2003; Kendler, 1980; Robins & Guze, 1970). Robins and Guze argued that psychiatric diagnoses should be based on systematic studies instead of “a priori principles” and defined five areas in which such studies should be carried out: 1) systematic clinical descriptions, 2) laboratory studies, 3) delineation from other disorders, 4) follow-up studies, and 5) family studies. 

Kendler (1980) added that diagnostic validity should require follow-up studies showing diagnostic consistency over time, similar rates of relapse and recovery, and homogeneous response to treatment, while family studies should show aggregation of similar symptom constellations among relatives. 

Andreasen (1995) declared that psychiatry had reached the stage where it was now “founded on diagnoses that are validated by clinical description and epidemiological criteria” and called for a “second structural program for the validation of psychiatric diagnoses” based on “methods that are being applied to track mental illnesses back to the organ system from which they emanate, the brain, and to the aberrations occurring at a molecular level in DNA”. 

Kendell and Jablensky (2003) attempted to emphasize the scientific basis for diagnostic classifications by separating validity from utility. They suggested that diagnostic categories “should be regarded as valid only if shown to be discrete entities with natural boundaries that separate them from other disorders” or from normality by a “zone of rarity”, or if defining characteristics, such as chromosome or biochemical abnormalities, delineate the diagnosis from other conditions with similar symptoms, and concluded that “most diagnostic concepts have not been shown to be valid in this sense”. 

When the DSM-5 was published in 2013, a wider public debate on the validity of psychiatric diagnoses ensued. The field trials had shown very poor test-retest reliability and prevelences for some disorders varied considerably by small changes in diagnostic criteria, such as the age when symptoms of ADHD should first have been apparent (as detailed in books by, among others, Greenberg and Frances). The NIMH then suggested an alternative model that allowed also subsets of functions and symptoms but required theoretical and/or empirical co-variates for each problem type (RDoC). In these debates, validity of diagnoses was often referred to as validity of assessment, and vice-versa. 

It may therefore be useful to go back to the original Robins and Guze criteria and re-assess them in view of recent empirical findings.

Clinical validity (Robins and Guze criteria 1 and 3)

Considering what we now know about the epidemiology of mental health problems, it is obvious that what Andreasen expected in 1995 has not come about. First, no mental disorder (besides mental symptoms induced by medical diseases, such as Huntington’s chorea) has yet been statistically distinguished from the normal variation by a “zone of rarity” or shown to constitute a “taxon” among other problem types in the population variance (Cloninger, 1999). Instead, the notions of “broader phenotypes” or “sub-threshold” disorders (initially described in relatives of probands in genetic research) and “spectra” of “overlapping” or “comorbid” disorders, have gained wider acceptance. 

Laboratory “markers” (Robins and Guze criterion 2)

Findings from the laboratory have provided no further support for the categorical system. Andreasen (1995) noted that the “markers” required by Robins and Guze “had not emerged” and that they had rather “risen and fallen” (e.g. the dexamethasone suppression test for depression), but her confidence in the development of new methods, such as brain imaging and molecular genetics, remained unbroken. Findings from studies using these increasingly sophisticated technical methods, however, have been at least as difficult to replicate, and/or as unspecific in relation to diagnostic categories, as those produced by the older models. Reports on new technologies to differentiate between a (small) group of patients and controls abound in the scientific literature, but no method with diagnostic specificity in relation to other problem types has yet been established.

Longitudinal follow-up (Robins and Guze criterion 4)

Research on longitudinal diagnostic stability is impeded by the artefactual hiatuses caused by the division in child- and adolescent psychiatry vs. adult psychiatry at about age 18, or in adult general psychiatry vs. “neuropsychiatry”. Child psychiatric conditions are often interpreted in terms of cognitive disabilities rather than disease, even in conditions for which medication is the standard treatment. With increasing age, definitions subsequently become more influenced by adult designations of symptoms, introducing concepts such as “paediatric” mania or “prodrome” schizophrenia. To what extent these clinical conditions really correspond to similar conditions in adulthood has not been established, but differences in symptom presentation and treatment responses seem to differ (e.g. SSRI treatment of depression in adolescents, Weller, Tucker, & Weller, 2005), and heterotypical progressions of problems from childhood into adulthood are the rule rather than the exceptions (Hofvander, Ossowski, Lundström, & Anckarsater, 2010).

Nor do longitudinal treatment effects seem to respect diagnostic categories. Pharmacological remedies alleviate symptoms across diagnostic divisions, no matter if their target is specific or wide. Lithium stabilizes mood in borderline personality disorder just as in bipolar disorder, atypical neuroleptics tranquilize, and serotonin reuptake inhibitors influence mood and anxiety regardless of diagnostics (Kramer, 1997). Psychotherapies and psychosocial interventions also have effects across diagnostic categories.

Familial aggregation (Robins and Guze criterion 5)

Familial aggregations of disorders have been studied by epidemiological methods to assess the overall importance of heritable factors for the variance in psychiatric phenomena. Family and adoption studies, not least twin studies, have provided ample support for the notion that hereditary factors play important causative roles in the variation of all mental health problems and associated features (Rutter & Silberg, 2002). This strand of research has used categorical as well as dimensional definitions (Levy, Hay, McStephen, Wood, & Waldman, 1997). More recent twin studies also collect data on co-existing and interacting problem constellations (Lichtenstein, et al., 2009) and follow developmental trajectories from adolescence (Silberg, Rutter, Neale, & Eaves, 2001) into the adulthood disorders (Cardno, Rijsdijk, Sham, Murray, & McGuffin, 2002; Kendler, Gardner, Annas, & Lichtenstein, 2008; Kendler, Gardner, & Prescott, 2003). Separate aetiologies have been reported for features previously linked into syndromes (Ronald, Happe, Price, Baron-Cohen, & Plomin, 2006), and conceptually different facets of clinical problem constellations have been found to have aetiological factors in common (Larsson, Andershed, & Lichtenstein, 2006).

Conclusion

None of diagnostic labels in use today meet the Robins & Guze criteria for validity. Psychiatrists are reluctant to recognize this, and often intertwine "valid assessments" with "valid diagnoses". A possible way forward would be to define "mental disorder" on the level of functioning and/or subjective suffering only, and then pursue research on symptom complexes, their aetiology and how they respond to treatment. Treatment efforts aimed at improving global functioning may also be evaluated on more relevant measures than reductions of some specific symptom cluster. But recognizing the lack of scientific validity for today's diagnoses may have far-reaching consequences for their use in legal contexts. 

This post is partly excerpted from Anckarsäter H. Beyond categorical diagnostics in psychiatry: scientific and medicolegal implications (2010). Should someone need a full-length manuscript or a reference list, don't hesitate to contact through the blog or e-mail henrik.anckarsater@neuro.gu.se.