HealthNewsReview.org | Independent Expert Reviews of News Stories | Holding Health and Medical Journalism Accountable |
|
|
New Test Aids Prognosis for Colon-Cancer Patients
May 15, 2009
Perhaps the most troubling example yet of early reporting of abstracts released far in advance of the American Society of Clinical Oncology meeting. Read this review to see why this is a big issue.
Our Review Summary
As the ratings show, this story about a new genetic test for colon cancer falls short of journalistic best practices in several ways. Among other things it fails to report costs, explain the results clearly, provide necessary caveats or put the test in context of clinical use.
Worse than that, viewed as a whole the story is simply misleading.
The current study demonstrates that the test, Genomic Health's Oncotype DK, can help identify patients at lower and higher risks of recurrence. But the study failed to show that the test could identify people more likely to benefit from chemotherapy.
Several other news reports treated these results with care: Sources in those stories are quoted saying that since the test can identify only higher and lower risk of recurrence, it might be useful as "a tie-breaker" in a yes-or-no-chemo decision; another says that it's "not as powerful as we'd like" in order to be of significant clinical value; a third that "it's a first step"; yet another calls it "half a loaf."
Several stories included in the lead or high up in the story the important fact that the test was not proven to identify people more likely to benefit from treatment.
Why did this story take such a different approach, allowing sources to imply the test has clinical value in the absence of evidence?
Possibly because the story didn’t include quotes from any truly independent sources. Given the fact that the study is unpublished and financed by the test maker, this is an alarming omission.
And what are the consequences of this sort of incomplete reporting?
After this story appeared in the printed Wall Street Journal in the morning, shortly after 3 p.m. the same afternoon the publication's Health Blog reported the following:
Supposedly, in some way, the early release of abstracts from the American Society of Clinical Oncoloy meeting, according to another story by the WSJ, was supposed to avoid stock manipulation.Genomic Health was trading higher on data suggesting its test for early-stage colon cancer may help patients and doctors decide whether they needed chemotherapy after their tumors are removed with surgery. Read the WSJ story on the colon-cancer test here.
We are troubled by the early release - which heightens speculation - and by the reporting on that speculation.
Click on Criteria for definitions.
The story says the company that makes the Oncotype DX test, Genomic Health, plans to start selling the test early next year.
But the reader has no idea how realistic this prediction is. Given the economic self-interest of the source, the story should have indicated what regulatory or commercial barriers the company faces before it can bring the test to market.
A company official declines to estimate how much the test will cost. This information is in the last line of the story.
It's tempting to assign a satisfactory rating for at least asking the question.
But a more complete story would have asked the official to estimate a price compared to the company's similar breast cancer test now on the market.
In fact, one did: A Bloomberg story reported that the company's breast cancer test costs about $3,800.
An even more complete story, from the Associated Press, mentioned that despite their price insurers often pay for some tests like this because they may be able to avoid spending $30,000 to $40,000 for chemotherapy on patients who do not benefit.
Reporting costs is important for at least two reasons:
- New technologies drive up the costs of healthcare. As the public debate over healthcare reform continues, it's important for the media to place stories about potential benefits in the context of potential costs.
- In this case, where the study's sponsor has such a clear economic interest in publicizing positive findings, readers should be alerted to how much money is at stake.
The reporter does a good job describing the prevalence, incidence and mortality of the disease without exaggeration.
The story states that the company has "unveiled" results of the study and they will be presented at an upcoming oncology conference. This is another story based on an incomplete abstract released far in advance of the American Society of Clinical Oncology meeting that hasn't even taken place yet.
The story fails to include the caveats that the study has not been peer-reviewed and has not been published.
In a study funded by the test's maker, it's essential to mention those facts.
Further, the study--a retrospective review of patients whose tumor biology was analyzed and a predictive model created from those analyses--lacks the power of a prospective clinical trial. The story should have said this.
Because the study failed to prove that the Oncotype DX test could identify the patients most likely to benefit from chemotherapy, the risk is that the test would be used for this purpose anyway.
This triggers the risk of "false reassurance"--that a resassuring test result would lead to no aggressive treatment yet a bad outcome.
The story fails to warn about the potential risks of clinical over-use and over-reliance on a test that may not improve survival rates.The story makes clear that the test is similar to the company's genetic test to identify more lethal breast cancers, yet that it is the first such test for colon cancer.
The story makes an astonishing mess of the key data:
After more than six years of tracking, the study found that 44% of the patients had tumors that had a 12% risk of recurrence within three years, which the researchers considered low risk. An additional 26% had a three-year recurrence risk of 22%, which researchers put in the high-risk group. The rest were in the middle.
Even a reader lucky enough to have within easy reach a calculator and a high school student taking advanced statistics is unlikely to figure out what this means in practical terms.
But this is what you get when you report on abstracts far in advance of any possible publication.
The story does not appear to draw excessively from the company's press release.
The story quotes three sources:
- an oncologist who has no ties to the study, but who leads the meeting where the study will be presented--and so has a bias in favor of the study's importance
- the lead author of the study
- the test maker's chief medical officer
Given the fact that [as the story discloses] the study was paid for by the test maker, the lack of independent sources is glaring.
The story would have been much stronger if it had comments from a disinterested oncologist who could speak to the test's practical value in a clinical setting.
The story explains that the current alternative option to the genetic test is to assess risk of recurrence based on a patient's age, other illnesses and tumor characteristics--a method that results in a clear recommendation for or against chemo in only 30 percent of patients.
Total Score: 4 of 10 Satisfactory
The U.S. Preventive Services Task Force is considered the gold standard of preventive health recommendations - including on screening tests. It's a good source for journalists and consumers.
About 70% of the stories reviewed from 2006-9 failed to adequately discuss costs, or to explain how big (or small) are the potential benefits and harms of treatments, tests, products and procedures.
We have documented a disturbing trend of news stories taking an advocacy stance, promoting certain screening tests outside the boundaries of scientific evidence.
Stories on new technologies like Cyberknife, DaVinci robotic surgery systems, and proton beam cancer therapy often fail to scrutinize the evidence and/or to discuss the costs involved.
Rather than suggesting that everyone should be screened for everything, news stories could explain: "All screening tests cause harm; some may do good."
The first 38 network TV network morning health news stories reviewed in 2009 earned an average score of 1.2 stars. 13 of the 38 stories got ZERO stars.
Both TIME magazine and BusinessWeek have published terrific stories explaining the importance of the Number Needed to Treat - or NNT.
Knowing relative risk reduction is like knowing you have a 50% off coupon but not knowing whether it's for a Lexus or a lollipop. Absolute risk reduction tells you what the "coupon" is worth. Read more.
The website NoFreeLunch.org posts "a database of health care professionals who have pledged to accept no gifts from industry and to rely on non-promotional sources of information."
To help journalists cover stories responsibly, we post a list of independent experts who state that they do not have financial ties to drug or medical device manufacturers.
We apply the same ten standardized criteria to the review of every story.
We have about 30 story reviewers. Each story is reviewed by 3 different people.
Gary Schwitzer's seven words you shouldn't use in medical news: cure, miracle, breakthrough, promising, dramatic, hope, victim. Read why.
Our reviewers include two former CNN medical reporters and a former editor of the Washington Post health section.