What is a mental "disorder" (part 3): validity of diagnoses according to the Robins & Guze criteria (1970)

In psychiatry, validity has mostly been an issue when determining how “validly” a diagnostic instrument may identify a categorical diagnosis. Comparatively less attention has been focused on the validity of the diagnostic constructs. 

A few leading psychiatrists have developed criteria for the validity of disorders (Andreasen, 1995; Kendell & Jablensky, 2003; Kendler, 1980; Robins & Guze, 1970). Robins and Guze argued that psychiatric diagnoses should be based on systematic studies instead of “a priori principles” and defined five areas in which such studies should be carried out: 1) systematic clinical descriptions, 2) laboratory studies, 3) delineation from other disorders, 4) follow-up studies, and 5) family studies. 

Kendler (1980) added that diagnostic validity should require follow-up studies showing diagnostic consistency over time, similar rates of relapse and recovery, and homogeneous response to treatment, while family studies should show aggregation of similar symptom constellations among relatives. 

Andreasen (1995) declared that psychiatry had reached the stage where it was now “founded on diagnoses that are validated by clinical description and epidemiological criteria” and called for a “second structural program for the validation of psychiatric diagnoses” based on “methods that are being applied to track mental illnesses back to the organ system from which they emanate, the brain, and to the aberrations occurring at a molecular level in DNA”. 

Kendell and Jablensky (2003) attempted to emphasize the scientific basis for diagnostic classifications by separating validity from utility. They suggested that diagnostic categories “should be regarded as valid only if shown to be discrete entities with natural boundaries that separate them from other disorders” or from normality by a “zone of rarity”, or if defining characteristics, such as chromosome or biochemical abnormalities, delineate the diagnosis from other conditions with similar symptoms, and concluded that “most diagnostic concepts have not been shown to be valid in this sense”. 

When the DSM-5 was published in 2013, a wider public debate on the validity of psychiatric diagnoses ensued. The field trials had shown very poor test-retest reliability and prevelences for some disorders varied considerably by small changes in diagnostic criteria, such as the age when symptoms of ADHD should first have been apparent (as detailed in books by, among others, Greenberg and Frances). The NIMH then suggested an alternative model that allowed also subsets of functions and symptoms but required theoretical and/or empirical co-variates for each problem type (RDoC). In these debates, validity of diagnoses was often referred to as validity of assessment, and vice-versa. 

It may therefore be useful to go back to the original Robins and Guze criteria and re-assess them in view of recent empirical findings.

Clinical validity (Robins and Guze criteria 1 and 3)

Considering what we now know about the epidemiology of mental health problems, it is obvious that what Andreasen expected in 1995 has not come about. First, no mental disorder (besides mental symptoms induced by medical diseases, such as Huntington’s chorea) has yet been statistically distinguished from the normal variation by a “zone of rarity” or shown to constitute a “taxon” among other problem types in the population variance (Cloninger, 1999). Instead, the notions of “broader phenotypes” or “sub-threshold” disorders (initially described in relatives of probands in genetic research) and “spectra” of “overlapping” or “comorbid” disorders, have gained wider acceptance. 

Laboratory “markers” (Robins and Guze criterion 2)

Findings from the laboratory have provided no further support for the categorical system. Andreasen (1995) noted that the “markers” required by Robins and Guze “had not emerged” and that they had rather “risen and fallen” (e.g. the dexamethasone suppression test for depression), but her confidence in the development of new methods, such as brain imaging and molecular genetics, remained unbroken. Findings from studies using these increasingly sophisticated technical methods, however, have been at least as difficult to replicate, and/or as unspecific in relation to diagnostic categories, as those produced by the older models. Reports on new technologies to differentiate between a (small) group of patients and controls abound in the scientific literature, but no method with diagnostic specificity in relation to other problem types has yet been established.

Longitudinal follow-up (Robins and Guze criterion 4)

Research on longitudinal diagnostic stability is impeded by the artefactual hiatuses caused by the division in child- and adolescent psychiatry vs. adult psychiatry at about age 18, or in adult general psychiatry vs. “neuropsychiatry”. Child psychiatric conditions are often interpreted in terms of cognitive disabilities rather than disease, even in conditions for which medication is the standard treatment. With increasing age, definitions subsequently become more influenced by adult designations of symptoms, introducing concepts such as “paediatric” mania or “prodrome” schizophrenia. To what extent these clinical conditions really correspond to similar conditions in adulthood has not been established, but differences in symptom presentation and treatment responses seem to differ (e.g. SSRI treatment of depression in adolescents, Weller, Tucker, & Weller, 2005), and heterotypical progressions of problems from childhood into adulthood are the rule rather than the exceptions (Hofvander, Ossowski, Lundström, & Anckarsater, 2010).

Nor do longitudinal treatment effects seem to respect diagnostic categories. Pharmacological remedies alleviate symptoms across diagnostic divisions, no matter if their target is specific or wide. Lithium stabilizes mood in borderline personality disorder just as in bipolar disorder, atypical neuroleptics tranquilize, and serotonin reuptake inhibitors influence mood and anxiety regardless of diagnostics (Kramer, 1997). Psychotherapies and psychosocial interventions also have effects across diagnostic categories.

Familial aggregation (Robins and Guze criterion 5)

Familial aggregations of disorders have been studied by epidemiological methods to assess the overall importance of heritable factors for the variance in psychiatric phenomena. Family and adoption studies, not least twin studies, have provided ample support for the notion that hereditary factors play important causative roles in the variation of all mental health problems and associated features (Rutter & Silberg, 2002). This strand of research has used categorical as well as dimensional definitions (Levy, Hay, McStephen, Wood, & Waldman, 1997). More recent twin studies also collect data on co-existing and interacting problem constellations (Lichtenstein, et al., 2009) and follow developmental trajectories from adolescence (Silberg, Rutter, Neale, & Eaves, 2001) into the adulthood disorders (Cardno, Rijsdijk, Sham, Murray, & McGuffin, 2002; Kendler, Gardner, Annas, & Lichtenstein, 2008; Kendler, Gardner, & Prescott, 2003). Separate aetiologies have been reported for features previously linked into syndromes (Ronald, Happe, Price, Baron-Cohen, & Plomin, 2006), and conceptually different facets of clinical problem constellations have been found to have aetiological factors in common (Larsson, Andershed, & Lichtenstein, 2006).

Conclusion

None of diagnostic labels in use today meet the Robins & Guze criteria for validity. Psychiatrists are reluctant to recognize this, and often intertwine "valid assessments" with "valid diagnoses". A possible way forward would be to define "mental disorder" on the level of functioning and/or subjective suffering only, and then pursue research on symptom complexes, their aetiology and how they respond to treatment. Treatment efforts aimed at improving global functioning may also be evaluated on more relevant measures than reductions of some specific symptom cluster. But recognizing the lack of scientific validity for today's diagnoses may have far-reaching consequences for their use in legal contexts. 

This post is partly excerpted from Anckarsäter H. Beyond categorical diagnostics in psychiatry: scientific and medicolegal implications (2010). Should someone need a full-length manuscript or a reference list, don't hesitate to contact through the blog or e-mail henrik.anckarsater@neuro.gu.se. 

Mental disorder is a cause of crime

The next couple of entries in the Forensic Friday series will continue the series on causation by addressing the notion that “mental disorder is a cause of crime”. This statement is really the foundation of forensic psychiatry. 

If mental disorders do not cause crimes, why should we have forensic psychiatry at all? After all, we don't have forensic internal medicine for diabetics who commit crimes, even if they may have done so during hypoglycaemia and need treatment while serving prison.

In coming posts, we will look at the historical roots of forensic psychiatry and adress each of the definitions “mental”, “disorder”, "cause" and “crime” conceptually, epistemologically, and empirically. The blog posts will be based on the paper "Mental disorder is a cause of crime" I co-authored with Susanna Radovic, Christer Svennerlind, Pontus Höglund and Filip Radovic in 2009. Without the collaborative work behind that paper, I wouldn't have been able to write any of the following texts. Many of them will be expanded and up-dated versions of paragraphs from the paper.  Of course, my version of philosphical analysis is over-simplified and not up to the level of my colleagues from philosophy, but I write this from the perspective of a clinician. On the other hand, I will go deeper into empirical data and try to sum up the considerable developments we have seen since 2009. 

Neither “mental”, “disorder” or “crime”, nor the “causation” implied, are clear-cut concepts. “Mental” denotes heterogeneous aspects of a person, such as inner experiences, cognitive abilities, and behaviour patterns, described in a non-physical vocabulary. In psychology and psychiatry, “mental” describes law-bound, caused aspects of human functioning that in some respect are predictable and generalizable. A corner-stone in these sciences is behaviour, as it is the easiest aspect to observe and quantify, and in itself contains a better-than-chance prediction of future behaviour. Indeed, even interviews and other more clearly “mental” assessments focus on reporting behaviours rather than “inner” mental processes.

Problems defined as “disorders” seem to be end-points of dimensional inter-individual differences rather than natural categories.

Deficits in cognitive faculties, such as attention, verbal understanding, and reality assessment, may be susceptibility factors that relate to behaviours, such as crimes, as risk factors in the sense of increasing the probability for a specific negative behaviour, and/or as causes in the sense of INUS conditions (insufficient but non-redundant parts of unnecessary but sufficient conditions).

Neurobiological measures (e.g. regional brain activity, neurotransmitters, genetics) may be related to crimes in a similar way as assessments of mental health problems or cognitive faculties.

Attributing causation to behaviours is not unbiased, and factors we feel estranged from, such as mental disorders, tend to attract disproportionate attention in explaining abject behaviours, such as crimes. 

When a forensic psychiatrist testifies in court, (s)he may describe mental health assessments, effects on accountability and ability, but the court and laypersons will understand what is said in terms of “causation”. Sometimes we are either lead to or volunteering to proffer an opinion on causation per se. I hope that this blog series will clarify what we can know and what we can’t know about whether mental disorders cause crimes based on psychiatry and psychology. To be continued.

Professional ethics in forensic psychiatry

Even if I work at the Centre for Ethics, Law and Mental Health (CELAM) at the University of Gothenburg, I am no specialist in ethics and have never studied practical philosophy. This post is thus not written by an expert in ethics, and I doubt that my colleagues who hold such qualifications would agree with much of it. Nevertheless, I want to argue for some positions I have arrived at through my work as a clinician and researcher. Forensic psychiatry is a medical speciality operating in the interface towards law. It is sometimes argued that we have a “dual task” of promoting our patients’ health and protecting society from violent (re-)offending. I reject this idea, and will outline the reasons for this here.

My position is that, as health professionals, our sole duty is to further the best interest of our patient. By health professionals, I refer to board licensed MDs, psychologists, psychotherapists, social workers, nurses and other professions employed in the health care system. Being a health professional requires that other people put their trust in you in very special ways. Therefore, we have ethical rules that must always be adhered to. To do our work, we can literally ask a fellow human to undress or to reveal his/her innermost secrets. Sooner or later, most of us will need a health professional. In the vulnerable position of the care seeker, and to get good treatment, we must be able to trust that the professional is dedicated to promoting our health and not serving two masters. 

Before going further into the arguments for my position, let me state that it is never in the patient’s best interest to commit a violent crime, having drugs prescribed without medical indication, false certificates issued or the similar.

Psychiatrists may give court expertise on evaluations and treatment, if the patient waives professional secrecy. In cases of diminished or incapacitated accountability, special legislation is at hand to safeguard the patient’s best interest. But the model in which a psychiatrist may “change hats” and give evidence against someone, working for the state or the prosecution, putting aside normal medical ethics, damages medicine at large. 

The reason the court asks for a psychiatric or psychological opinion is that the expert has specialist training (based in the confidence of patients and research subjects) and clinical knowledge (that demands access to information that the patient has provided under the impression of confidentiality, either directly to the expert or to other experts, who have entered it into files). Being a qualified health professional also brings credibility and power imbalance, making it difficult for the person described to question the expert’s opinion.

It does not suffice to merely state that “this opinion is made as expert for XXX” as established titles, such as MD, will create certain expectations in those receiving the evaluation. In contrast, it could be possible for a psychologist to work as a behavioual analyst and give expertise in that role. Such cases have to be handled with great caution, however, in order not to confuse previous or future clients.  

But with these requirements, how could medical doctors work in forensic medicine or with societal response to contagious diseases? Frankly, I do not know how ethical frame-works, especially for the latter, have been developed, but I do not find clear analogies to forensic psychiatry. Psychiatry has a rather weak inter-rater reliability, and courts tend to believe us more easily than our colleagues do, who often disagree with our opinions. And without trust from patients, we are completely lost. So I do not find the responsibilities and benefits of assuming the role of “(s)he who knows” in the courtroom worth the risks for our profession. 

Not doing harm is our first duty. How is that possible when giving expert opinions in situations where we have no control over how the expertise is used, disseminated or understood? Instead, I think that we should use our scarce resources to develop good health services to incarcerated or otherwise sentenced persons, evidence-based treatments for aggressive behaviours and public mental health education.