Health News Review

Perhaps the most troubling example yet of early reporting of abstracts released far in advance of the American Society of Clinical Oncology meeting. Read this review to see why this is a big issue.

Our Review Summary

As the ratings show, this story about a new genetic test for colon cancer falls short of journalistic best practices in several ways. Among other things it fails to report costs, explain the results clearly, provide necessary caveats or put the test in context of clinical use.

Worse than that, viewed as a whole the story is simply misleading.

The current study demonstrates that the test, Genomic Health’s Oncotype DK, can help identify patients at lower and higher risks of recurrence. But the study failed to show that the test could identify people more likely to benefit from chemotherapy.

Several other news reports treated these results with care: Sources in those stories are quoted saying that since the test can identify only higher and lower risk of recurrence, it might be useful as "a tie-breaker" in a yes-or-no-chemo decision; another says that it’s "not as powerful as we’d like" in order to be of significant clinical value; a third that "it’s a first step"; yet another calls it "half a loaf."

Several stories included in the lead or high up in the story the important fact that the test was not proven to identify people more likely to benefit from treatment.

Why did this story take such a different approach, allowing sources to imply the test has clinical value in the absence of evidence?

Possibly because the story didn’t include quotes from any truly independent sources. Given the fact that the study is unpublished and financed by the test maker, this is an alarming omission.

And what are the consequences of this sort of incomplete reporting?

After this story appeared in the printed Wall Street Journal in the morning, shortly after 3 p.m. the same afternoon the publication’s Health Blog reported the following: 

Genomic Health was trading higher on data suggesting its test for early-stage colon cancer may help patients and doctors decide whether they needed chemotherapy after their tumors are removed with surgery. Read the WSJ story on the colon-cancer test here.

Supposedly, in some way, the early release of abstracts from the American Society of Clinical Oncoloy meeting, according to another story by the WSJ, was supposed to avoid stock manipulation.  

We are troubled by the early release – which heightens speculation – and by the reporting on that speculation.


Criteria

Not Satisfactory

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

A company official declines to estimate how much the test will cost. This information is in the last line of the story. 

It’s tempting to assign a satisfactory rating for at least asking the question. 

But a more complete story would have asked the official to estimate a price compared to the company’s similar breast cancer test now on the market.

In fact, one did: A Bloomberg story reported that the company’s breast cancer test costs about $3,800.

An even more complete story, from the Associated Press, mentioned that despite their price insurers often pay for some tests like this because they may be able to avoid spending $30,000 to $40,000 for chemotherapy on patients who do not benefit. 

Reporting costs is important for at least two reasons: 

  • New technologies drive up the costs of healthcare. As the public debate over healthcare reform continues, it’s important for the media to place stories about potential benefits in the context of potential costs.
  • In this case, where the study’s sponsor has such a clear economic interest in publicizing positive findings, readers should be alerted to how much money is at stake.
Not Satisfactory

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story makes an astonishing mess of the key data:

After more than six years of tracking, the study found that 44% of the patients had tumors that had a 12% risk of recurrence within three years, which the researchers considered low risk. An additional 26% had a three-year recurrence risk of 22%, which researchers put in the high-risk group. The rest were in the middle.

Even a reader lucky enough to have within easy reach a calculator and a high school student taking advanced statistics is unlikely to figure out what this means in practical terms.

But this is what you get when you report on abstracts far in advance of any possible publication. 

Not Satisfactory

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Because the study failed to prove that the Oncotype DX test could identify the patients most likely to benefit from chemotherapy, the risk is that the test would be used for this purpose anyway. 

This triggers the risk of "false reassurance"–that a resassuring test result would lead to no aggressive treatment yet a bad outcome. 

The story fails to warn about the potential risks of clinical over-use and over-reliance on a test that may not improve survival rates.

Not Satisfactory

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story states that the company has "unveiled" results of the study and they will be presented at an upcoming oncology conference.  This is another story based on an incomplete abstract released far in advance of the American Society of Clinical Oncology meeting that hasn’t even taken place yet. 

The story fails to include the caveats that the study has not been peer-reviewed and has not been published.

In a study funded by the test’s maker, it’s essential to mention those facts.

Further, the study–a retrospective review of patients whose tumor biology was analyzed and a predictive model created from those analyses–lacks the power of a prospective clinical trial. The story should have said this.

Satisfactory

Does the story commit disease-mongering?

Satisfactory

The reporter does a good job describing the prevalence, incidence and mortality of the disease without exaggeration. 

Not Satisfactory

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story quotes three sources:

  • an oncologist who has no ties to the study, but who leads the meeting where the study will be presented–and so has a bias in favor of the study’s importance
  • the lead author of the study
  • the test maker’s chief medical officer

Given the fact that [as the story discloses] the study was paid for by the test maker, the lack of independent sources is glaring.

The story would have been much stronger if it had comments from a disinterested oncologist who could speak to the test’s practical value in a clinical setting.

Satisfactory

Does the story compare the new approach with existing alternatives?

Satisfactory

The story explains that the current alternative option to the genetic test is to assess risk of recurrence based on a patient’s age, other illnesses and tumor characteristics–a method that results in a clear recommendation for or against chemo in only 30 percent of patients.

 

Not Satisfactory

Does the story establish the availability of the treatment/test/product/procedure?

Not Satisfactory

The story says the company that makes the Oncotype DX test, Genomic Health, plans to start selling the test early next year.

But the reader has no idea how realistic this prediction is. Given the economic self-interest of the source, the story should have indicated what regulatory or commercial barriers the company faces before it can bring the test to market. 

Satisfactory

Does the story establish the true novelty of the approach?

Satisfactory

The story makes clear that the test is similar to the company’s genetic test to identify more lethal breast cancers, yet that it is the first such test for colon cancer.

Not Applicable

Does the story appear to rely solely or largely on a news release?

Not Applicable

The story does not appear to draw excessively from the company’s press release.

Total Score: 3 of 9 Satisfactory


We Welcome Comments

But please note: We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, profanity or any from anyone who doesn't list what appears to be an actual email address. We will also end any thread of repetitive comments. We don't give medical advice so we won't respond to questions asking for it. Please see more on our comments policy.