HealthNewsReview.org | Independent Expert Reviews of News Stories | Holding Health and Medical Journalism Accountable |
|
|
Breakthrough for MS?
February 10, 2009
Strength: it emphasized this was a small, early study. But it focused exclusively on one patient with an excellent outcome, giving no broader data from the study. No independent sources were quoted.
Our Review Summary
This TV segment, based on results of an early stage clinical trial of a stem cell treatment for MS, suffers from a major structural flaw: It focuses exclusively on the case of a single patient with an excellent outcome, without mentioning findings about the larger study group. In fact, it doesn't even say that the study was published.
Given the magnitude of that failure, and the story's resulting capacity to mislead viewers, other flaws are nearly irrelevant. Still, it's worth mentioning that no independent sources were interviewed to put the case in context.
The story does manage to do a few things well:
- It carefully discloses that the study was small and the findings preliminary and in need of confirmation.
- It does not promise imminent availability of the treatment.
- During the studio interview, the host makes a useful clarification that the patient is not "cured" of MS but is currently free of symptoms.
It's hard to imagine why the producers decided to focus on the case of a single outlier without even acknowledging the rest of the findings, or even that they are published. One fears that it's done to create the impression of "exclusivity"--that CBS alone had this story. If this is the reason, the viewers' ignorance about the larger study findings, and their limitations, is serious collateral damage.
It's worth noting that despite the hosts' efforts to explain that the study is small and the results preliminary, some visitors to the CBS online comment area wanted to know where they could get the treatment. It's a useful reminder that stories that emphasize positive outcomes may provoke irrational hope.
Which in turn should remind for journalists to present scientific findings as fairly and accurately as possible--even when they've landed a memorable interview.
Click on Criteria for definitions.
The story makes clear the stem cell treatment for MS is still under study and not yet available.
It would have been useful to tell people whether the larger randomized trail is still recruiting patients and, if so, how to get more information.
This story mentioned several times that this was a small early trial - so we think it's OK that they didn't discuss projected costs of this treatment.
The segment uses one anecdote at length, which describes a single patient's case of MS from diagnosis to outcome. Because he is young, his prognosis poor and the outcome of the stem cell treatment so apparently successful, this could be misleading.
Having said that, the report does not exaggerate the prevalence or severity of the disease, so it minimally earns a "satisfactory" rating.
The segment is based on results of a small, preliminary clinical trial. The reporter discloses this, and indicates that a larger, randomized clinical trial will be required to verify the findings.
Oddly, however, the segment fails to mention that results of this trial have been published in a medical journal, The Lancet Neurology. This could lead the viewer to believe the study is less credible than it is.
Further, the segment focuses on one patient so thoroughly that it implies the one case is evidence of treatment efficacy.
Finally, the report fails to mention other key limitations on the evidence, including that the study looked only at people with the early stage of the disease, that an earlier trial on patients with more advanced cases had negative results and that the long-term outcomes are unknown.
The story fails to mention potential harms related to the procedure, including those related to chemotherapy and the disabling of the immune system.
The story fails to note that previous trials have employed stem cell treatments for MS. The novelty of this approach is the use of a priming process that is less toxic than the previously used chemotherapy protocols.
The story fails to deliver any detail about the benefits of the treatment other than the single anecdote.
Consequently, viewers have no idea what outcomes the other 20 patients had.
The published study shows good results for 17 of the 21 patients, with all at least stabilzing and most showing improvement by at least one point on a commonly used scale of MS symptoms. Some patients required a second treatment. The story should have said this.
Instead of reporting these details about the study, the producers allow viewers to assume that the featured patient's results are typical. It appears he is an outlier, the example of the best and most successful outcome of the study. The segment should have said this too.
A press release from Northwestern University was published on February 3, seven days before the Early Show story appeared.
The segment does not draw excessively from the press release.
The segment is built around a single patient [with his mother, during the studio interview portion] and the researcher conducting the trial. No other sources are consulted.
At least one independent expert in MS treatment should have been consulted to provide perspective.
The segment says that this patient was given conventional drugs at first, implying drugs are the first treatment. This is correct and useful.
Other treatments are used for MS, including deep-brain stimulation, though it is not approved for use on MS.
The segment would have been stronger with more information about current treatments, but it minimally reaches "satisfactory" status under this criterion.
Total Score: 4 of 9 Satisfactory
The U.S. Preventive Services Task Force is considered the gold standard of preventive health recommendations - including on screening tests. It's a good source for journalists and consumers.
About 70% of the stories reviewed from 2006-9 failed to adequately discuss costs, or to explain how big (or small) are the potential benefits and harms of treatments, tests, products and procedures.
We have documented a disturbing trend of news stories taking an advocacy stance, promoting certain screening tests outside the boundaries of scientific evidence.
Stories on new technologies like Cyberknife, DaVinci robotic surgery systems, and proton beam cancer therapy often fail to scrutinize the evidence and/or to discuss the costs involved.
Rather than suggesting that everyone should be screened for everything, news stories could explain: "All screening tests cause harm; some may do good."
The first 38 network TV network morning health news stories reviewed in 2009 earned an average score of 1.2 stars. 13 of the 38 stories got ZERO stars.
Both TIME magazine and BusinessWeek have published terrific stories explaining the importance of the Number Needed to Treat - or NNT.
Knowing relative risk reduction is like knowing you have a 50% off coupon but not knowing whether it's for a Lexus or a lollipop. Absolute risk reduction tells you what the "coupon" is worth. Read more.
The website NoFreeLunch.org posts "a database of health care professionals who have pledged to accept no gifts from industry and to rely on non-promotional sources of information."
To help journalists cover stories responsibly, we post a list of independent experts who state that they do not have financial ties to drug or medical device manufacturers.
We apply the same ten standardized criteria to the review of every story.
We have about 30 story reviewers. Each story is reviewed by 3 different people.
Gary Schwitzer's seven words you shouldn't use in medical news: cure, miracle, breakthrough, promising, dramatic, hope, victim. Read why.
Our reviewers include two former CNN medical reporters and a former editor of the Washington Post health section.