Penn team finds success with talk therapy for schizophrenics

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The story appears to deal with cost by including an estimate used by the researchers that treatment for people with schizophrenia costs $26,000 to $31,000 per year on average. However, there is no estimate of the cost of the weekly cognitive therapy sessions that are the central point of the story. Also, the story includes a claim by the lead researcher that this therapy would replace and be more effective than current therapy, and gave it credit for preventing an expensive hospitalization, but cost-comparisons were not part of the experiment nor is any other evidence provided in the story to support the claims.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story presents a series of success stories that gives readers a one-sided view of the results and glosses over the limitations of the design of this trial and the narrow selection of patients with extreme symptoms that raise questions about the conclusions and the relevance of these results to typical patients. The benefits reported in this trial are not quantified.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Perhaps the only potential harm of cognitive therapy would be wasting time and money on something that turns out not to produce the desired benefits. Still, since resources for mental health treatment are extremely limited and often not well covered by private insurance plans, the story could have addressed this potential downside.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

As noted above, the story does not make clear that the patients included in this trial were not average patients. A more serious shortcoming is the failure to explain to readers that there was no “placebo” control treatment or that with this sort of treatment it is not possible to blind the clinician to the type of treatment the patient is receiving. The researchers noted these important limitations in their journal article: “The [cognitive therapy] condition involved more patient contact than the [standard treatment] condition, raising the possibility that nonspecific patient contact factors are contributing to the observed group differences. Additionally, both therapists and patients were aware of the condition and participation in an experiment, introducing possible bias in the reported outcomes.” Readers should have been told that this trial does not actually prove that cognitive therapy alone explains the observed improvements. It could be that getting a weekly session with a PhD therapist using another type of approach could also produce benefits; that’s a question this trial did not address.

This trial was done at just a single center. The therapists were more highly trained than therapists seen in routine clinical care. Both the duration of the trial and the outcome measured used were changed during the study. These are red flags raising questions about the quality of the science and the relevance to routine clinical practice.

This story presents the results as far more conclusive than the study design warrants.

Does the story commit disease-mongering?

Not Satisfactory

This trial included only low-functioning patients with problems much more severe than the average patient. In their journal article the researchers described them as “patients with chronic schizophrenia selected from the extreme end of the low-functioning continuum,” but the story implies that the therapy could benefit anyone with schizophrenia. There were other special characteristics of the study population which raise questions about whether the results apply to the general population: two-thirds of the participants were African-American, and men outnumbered women 2-to-1 (a more extreme difference than seen in the general population). Also, the researchers ended up randomizing only a third of the patients initially invited to participate, further raising questions about how relevant the result are to average patients.