The story delivers multiple success stories, but fails to point out serious limitations in the design of the trial (single-center, no comparable “placebo” control and no blinding, for example) that raise questions about the claims made by researchers. While the journal article reporting the trial results highlights that the participants were selected from the “extreme end of the low-functioning continuum,” the story implies that the tested therapy would be beneficial to anyone with schizophrenia.
Experimental results are only as good as the design of the trial that produced them. This story should have highlighted study limitations that call into question the conclusions of this test of cognitive therapy. People with schizophrenia often have shortened lives and very high costs for treatments including powerful drugs and other intensive interventions. Effective psychotherapy would be a welcome addition to treatment options.
The story appears to deal with cost by including an estimate used by the researchers that treatment for people with schizophrenia costs $26,000 to $31,000 per year on average. However, there is no estimate of the cost of the weekly cognitive therapy sessions that are the central point of the story. Also, the story includes a claim by the lead researcher that this therapy would replace and be more effective than current therapy, and gave it credit for preventing an expensive hospitalization, but cost-comparisons were not part of the experiment nor is any other evidence provided in the story to support the claims.
The story presents a series of success stories that gives readers a one-sided view of the results and glosses over the limitations of the design of this trial and the narrow selection of patients with extreme symptoms that raise questions about the conclusions and the relevance of these results to typical patients. The benefits reported in this trial are not quantified.
Perhaps the only potential harm of cognitive therapy would be wasting time and money on something that turns out not to produce the desired benefits. Still, since resources for mental health treatment are extremely limited and often not well covered by private insurance plans, the story could have addressed this potential downside.
As noted above, the story does not make clear that the patients included in this trial were not average patients. A more serious shortcoming is the failure to explain to readers that there was no “placebo” control treatment or that with this sort of treatment it is not possible to blind the clinician to the type of treatment the patient is receiving. The researchers noted these important limitations in their journal article: “The [cognitive therapy] condition involved more patient contact than the [standard treatment] condition, raising the possibility that nonspecific patient contact factors are contributing to the observed group differences. Additionally, both therapists and patients were aware of the condition and participation in an experiment, introducing possible bias in the reported outcomes.” Readers should have been told that this trial does not actually prove that cognitive therapy alone explains the observed improvements. It could be that getting a weekly session with a PhD therapist using another type of approach could also produce benefits; that’s a question this trial did not address.
This trial was done at just a single center. The therapists were more highly trained than therapists seen in routine clinical care. Both the duration of the trial and the outcome measured used were changed during the study. These are red flags raising questions about the quality of the science and the relevance to routine clinical practice.
This story presents the results as far more conclusive than the study design warrants.
This trial included only low-functioning patients with problems much more severe than the average patient. In their journal article the researchers described them as “patients with chronic schizophrenia selected from the extreme end of the low-functioning continuum,” but the story implies that the therapy could benefit anyone with schizophrenia. There were other special characteristics of the study population which raise questions about whether the results apply to the general population: two-thirds of the participants were African-American, and men outnumbered women 2-to-1 (a more extreme difference than seen in the general population). Also, the researchers ended up randomizing only a third of the patients initially invited to participate, further raising questions about how relevant the result are to average patients.
The story does include independent sources. And while the story does not mention who funded the trial, the disclosure section of the journal article does not reveal obvious serious conflicts. However, it appears that all of the quoted experts use cognitive therapy in their practices, which may have limited the critical examination of the limitations of this trial.
The story mentions some of the strengths and weaknesses of drug treatment for schizophrenia. However, it gives the reader no sense of the magnitude. “Most medications affect the delusions and hallucinations”…. but how much of an effect?
The article notes that it is difficult to find therapists trained in cognitive therapy.
The story puts this trial in the context of a half-century of study of cognitive therapy. It reports that testing cognitive therapy with this type of patient is new.
The story does not appear to rely on a news release.
Systematic, Criteria-Driven
Monday at #ADA2021, an infectious disease epidemiologist, a journalist, and a researcher shared perspectives on communication of risk in the era of COVID-19: http://ow.ly/ROfR30rMeE7 @AstleyCM @garyschwitzer @berthahidalgo @ADA_DiabetesPro
Shoutouts to @susan_bewley @drkevinknopf @MichaelBaum11 in this call for more than fawning coverage from press releases for these kinds of stories.
Also, linked history inside goes back 5 years just on Grail apparently trying to become Holy Grail. https://twitter.com/garyschwitzer/status/1408849696416841731
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like