The story delivers multiple success stories, but fails to point out serious limitations in the design of the trial (single-center, no comparable “placebo” control and no blinding, for example) that raise questions about the claims made by researchers. While the journal article reporting the trial results highlights that the participants were selected from the “extreme end of the low-functioning continuum,” the story implies that the tested therapy would be beneficial to anyone with schizophrenia.
Experimental results are only as good as the design of the trial that produced them. This story should have highlighted study limitations that call into question the conclusions of this test of cognitive therapy. People with schizophrenia often have shortened lives and very high costs for treatments including powerful drugs and other intensive interventions. Effective psychotherapy would be a welcome addition to treatment options.
The story appears to deal with cost by including an estimate used by the researchers that treatment for people with schizophrenia costs $26,000 to $31,000 per year on average. However, there is no estimate of the cost of the weekly cognitive therapy sessions that are the central point of the story. Also, the story includes a claim by the lead researcher that this therapy would replace and be more effective than current therapy, and gave it credit for preventing an expensive hospitalization, but cost-comparisons were not part of the experiment nor is any other evidence provided in the story to support the claims.
The story presents a series of success stories that gives readers a one-sided view of the results and glosses over the limitations of the design of this trial and the narrow selection of patients with extreme symptoms that raise questions about the conclusions and the relevance of these results to typical patients. The benefits reported in this trial are not quantified.
Perhaps the only potential harm of cognitive therapy would be wasting time and money on something that turns out not to produce the desired benefits. Still, since resources for mental health treatment are extremely limited and often not well covered by private insurance plans, the story could have addressed this potential downside.
As noted above, the story does not make clear that the patients included in this trial were not average patients. A more serious shortcoming is the failure to explain to readers that there was no “placebo” control treatment or that with this sort of treatment it is not possible to blind the clinician to the type of treatment the patient is receiving. The researchers noted these important limitations in their journal article: “The [cognitive therapy] condition involved more patient contact than the [standard treatment] condition, raising the possibility that nonspecific patient contact factors are contributing to the observed group differences. Additionally, both therapists and patients were aware of the condition and participation in an experiment, introducing possible bias in the reported outcomes.” Readers should have been told that this trial does not actually prove that cognitive therapy alone explains the observed improvements. It could be that getting a weekly session with a PhD therapist using another type of approach could also produce benefits; that’s a question this trial did not address.
This trial was done at just a single center. The therapists were more highly trained than therapists seen in routine clinical care. Both the duration of the trial and the outcome measured used were changed during the study. These are red flags raising questions about the quality of the science and the relevance to routine clinical practice.
This story presents the results as far more conclusive than the study design warrants.
This trial included only low-functioning patients with problems much more severe than the average patient. In their journal article the researchers described them as “patients with chronic schizophrenia selected from the extreme end of the low-functioning continuum,” but the story implies that the therapy could benefit anyone with schizophrenia. There were other special characteristics of the study population which raise questions about whether the results apply to the general population: two-thirds of the participants were African-American, and men outnumbered women 2-to-1 (a more extreme difference than seen in the general population). Also, the researchers ended up randomizing only a third of the patients initially invited to participate, further raising questions about how relevant the result are to average patients.
The story does include independent sources. And while the story does not mention who funded the trial, the disclosure section of the journal article does not reveal obvious serious conflicts. However, it appears that all of the quoted experts use cognitive therapy in their practices, which may have limited the critical examination of the limitations of this trial.
The story mentions some of the strengths and weaknesses of drug treatment for schizophrenia. However, it gives the reader no sense of the magnitude. “Most medications affect the delusions and hallucinations”…. but how much of an effect?
The article notes that it is difficult to find therapists trained in cognitive therapy.
The story puts this trial in the context of a half-century of study of cognitive therapy. It reports that testing cognitive therapy with this type of patient is new.
The story does not appear to rely on a news release.