New DNA Tests Aimed at Reducing Colon Cancer

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The story says, “Both tests would be less expensive than colonoscopy, and potentially more effective.”  No specific cost estimate is given.  The Philadelphia Inquirer said “Though the cost has not been set, it could be as low as $300 per test.” Reuters reported that the company said earlier that it would aim for $300-400 per test.

The story also states that “Compliance with colonoscopy is low, since people don’t want to have one, and the overall cost per detection is high” – implying that the new test might be more cost-effective.  But it’s unclear at this point if that would be true. For example, previous stool DNA tests have performed much worse (higher costs, lower efficacy) than colonoscopy in modeling studies.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

Mixed bag. The story did provide some important context about test. For example, it says, “Exact Sciences reported in July that its test was highly sensitive and specific when applied directly to cells taken from tumors. But in the real world, the tumor DNA must be detected in stool samples, even though almost all the DNA comes from the bacteria of the gut. Just 0.01 percent of the DNA is human, and most of this is normal DNA, not the altered DNA of tumors.”

But only in passing does it mention that “in the real world, the tumor DNA must be detected in stool samples.”  It could/should have driven home the point more clearly that the methodology used in the study tends to overestimate performance compared with an evaluation done in a representative sample population (hence the need to do another study). WebMD, for example, did a better job on this point, with a quote from Dr. Durado Brooks of the American Cancer Society: “This was a very preliminary study that utilized samples from individuals who were known to have colon polyps or cancer.  The performance may decrease considerably when the test is used in a large population of healthy individuals.” Health Day quoted Brooks: “Showing that in a small group of samples is very different from demonstrating that in a population where only a small number of individuals are going to have polyps of that size.  Then we will know if this is a big step forward.”

There was also no discussion of the limitations of drawing conclusions from an abstract presented at a scientific conference – something that has not yet undergone the kind of peer review that a journal would use.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

This story relies too heavily on researchers who are connected to the companies developing the cancer screens. Where was the truly independent perspective?

Does the story compare the new approach with existing alternatives?

Not Satisfactory

As noted in the “benefits” criterion above, this story did not even discuss other stool tests like fecal immunochemical testing or other fecal occult blood tests that are currently available and have been shown to perform well in appropriate samples.  It did a decent job in comparing two new technologies, but got hung up on this comparison while forgetting existing tests.  In so doing, the end of the story devolves a bit into a “my product is better than your product” tit for tat. Which is what can happen when your only quoted sources all have connections to the companies involved.