Note to our followers: Due to a lack of sufficient funding, HealthNewsReview.org will cease daily publication of new content at the end of 2018. Publisher Gary Schwitzer and other contributors may post new articles periodically. If you wish to donate, your gift might help keep the site available to the public for a few more years, by defraying costs of web hosting and maintenance. All of our 6,000+ published articles contain lessons to help people improve their critical thinking about health care. Read more about our change in status. And here's how to make a donation.
Read Original Release

Stomach cancer breath test called “85% accurate overall” — is that accurate?

Breath test could help detect stomach and esophageal cancers

Our Review Summary

This news release about a breath test to detect stomach and esophageal cancers discusses the potential benefits of a screening test that could distinguish useful indications of early-stage, but serious, cancer in the breaths of average people –.if future research shows it can work in the real world. But that’s a bit off track from what the researchers tested. The underlying trial merely showed that in most cases breath analysis could distinguish between people who were already known to have stomach or esophageal cancer (advanced in most cases) with people who did not have any signs of cancer.

The release did note at the end that researchers have yet to study whether this breath analysis can detect cancers among people in the general population. That’s an important point and one that would have been good to emphasize earlier in the release. The release gave only the briefest nod to the existence of numerous other breath tests for cancer that are being studied and didn’t explain how this particular research compares to other efforts.

 

Why This Matters

While this research is a necessary step, there is scant support for the claims in the release that this screening test “could detect cancer” in apparently healthy people and the suggestion that the test could mean “earlier diagnosis and treatment, and better survival.” Those claims dominated news coverage of the research announcement, as seen here, here and here.

Detecting early cancer and then showing that early detection lowers stomach cancer mortality has not been demonstrated.

HealthNewsReview.org also reviewed a HealthDay story on the breath test study findings.

Criteria

Does the news release adequately discuss the costs of the intervention?

Not Satisfactory

This research is only at its earliest stages, so is it too early to expect cost information? If it’s not too early to promote a technique in a news release it’s not too early to provide a ballpark cost. The release does tout potential savings from avoiding “unnecessary” endoscopies but without also considering that an easy test could send many healthy into further testing or even treatment they didn’t need.

A 20% false positive rate, which this test apparently offers, could translate into a huge number of expensive endoscopies.

Does the news release adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The release claims benefits, including early detection, avoidance of unnecessary endoscopies and more successful treatment that were not actually looked at in the underlying study. In essence, the release speculated about an ideal, proven breath test, while the research report merely indicated that the results of a preliminary trial suggest that the concept would be worth trying in conditions that more closely resemble the real world. The release should have noted that even if such a breath test could reliably detect these cancers at an earlier stage that does not necessarily mean patients would live longer. It could be that deadly stomach and esophageal cancers are resistant to treatment early in their course, so that early detection would merely provide bad news sooner. The effectiveness of treatments is a separate question.

The claim that the test is “85% accurate overall” also merits scrutiny. What appears to be an accurate test in a small group in which half of them have the disease, turns out to be not so favorable when you screen a normal population in which the disease prevalence is low. For instance, with a specificity of 80%, 20% of those without cancer will have a positive test (false positive rate of 20%). If you have 300 people (as in the study) there will be 60 false positives. In the study there were 150 people with cancer and 124 will have a positive test (80% sensitivity). So out of 184 positive tests, 124 will actually have cancer. That is a positive predictive value of 67% (assuming if I test positive, I have a 2/3 chance of having cancer).

Now take a hypothetical population where the prevalence of cancer is 1/1000. There will be one person with cancer (they will probably have a positive test) but there will be 200 healthy people, without cancer, who will also have a positive test! In that case only 1 out of 201 positive tests is actually a person with cancer and the PPV is .005 or less than half of one percent. If the test is positive you have a 1/201 chance of having cancer (the other 200 people don’t have cancer!). And those 200 other people have probably suffered some harm by having a false positive test.

Does the news release adequately explain/quantify the harms of the intervention?

Not Satisfactory

The release makes no mention of the harms of such a breath test. If the release is going to discuss potential unproven benefits, it should also mention the potential harms of screening tests including false positives, false negatives leading to over- or under-diagnosis.

Chief among these harms would be falsely labeling healthy people as possibly having cancer and then subjecting them to invasive testing or even treatments that turn out to be unnecessary. Based on the statistics in the study abstract, an 81% specificity suggests that almost one-out-of-five people taking the test would be labeled as possibly having cancer when they don’t. However, the 80% sensitivity suggests that at present about one-in-five people with cancer would be missed. (Computed tomography (CT) screening for lung cancer is about that). The numbers get much worse, as discussed above, when we consider using the test in the general population of people who aren’t likely to have stomach cancer.

Does the news release seem to grasp the quality of the evidence?

Not Satisfactory

The release suggests that the results of this trial provides evidence that a breath test could detect early stage cancer. However, that is not what the trial tested. The cancer patients in the trial had already been definitively diagnosed by conventional methods and most had advanced cancers. Since none of the participants had an early cancer that would not be detected by conventional methods, it is too early to say anything about the ability of such a breath test to detect early stage cancer.

The trial abstract made clear that what researchers found was that about 85% of the time they could distinguish between a healthy person and a person with known cancer — but it then clearly noted that the breath test still has to be tried with participants who are more like the people who would actually be candidates for such a test in the future, if further research goes well. It is only near the very end of the release that it is noted that researchers still need to “assess the ability of the test to pick up cases within a group that is likely to contain only a small percentage of cancers.” We wish that important point had been emphasized, and much earlier in the release.

Does the news release commit disease-mongering?

Satisfactory

The release doesn’t engage in disease mongering and provides statistics on the numbers of stomach and esophageal cancers diagnosed globally each year, which matches other sources, and this puts the disease in context for readers. But when it notes that these types of cancers have a low 5-year survival rate of 15%, it would have been better to also tell readers that, while deadly, these two cancers are relatively uncommon (accounting for less than 10% of cancer diagnoses even after excluding common, nonmelanoma skin cancer) and appear to be largely preventable through improvements in diet and other living conditions.

Does the news release identify funding sources & disclose conflicts of interest?

Satisfactory

Funders are named in a sidebar to the release on the EurekAlert! website. Ideally, the funders would also be listed in the release text so that when the release is republished that information is available.

The study abstract noted that the researchers stated they had no conflicts to declare.

Does the news release compare the new approach with existing alternatives?

Not Satisfactory

The release seems to be favorably comparing this breath test, which is a screening test, to conventional diagnostic methods, including endoscopy, even though there is not yet any evidence to support comparisons.

The release would have been improved with a more careful distinction of who the screening test is geared for — is it a screening tool for healthy people to help rule out cancer or is it only for those with cancer symptoms?

Does the news release establish the availability of the treatment/test/product/procedure?

Satisfactory

It is clear that the breath test is not available. The release did not make any predictions about when such a test might make its way through the full course of experimental trials.

Does the news release establish the true novelty of the approach?

Satisfactory

We’ll give the release a pass on this since a breath test for screening for these specific cancers would be a new development, if proven effective, and the release does reference other research that formed the foundation for the new study.

However, the release did not explain how this research is distinct from other work in the area. It would have been helpful had the release noted that many research teams around the world are evaluating breath tests to detect various types of cancer and other diseases. A report out of Israel in 2015 sparked wide news coverage of research into a breath test that appeared to be able to distinguish stomach cancer from pre-cancerous lesions, many of which do not develop into cancer.

Does the news release include unjustifiable, sensational language, including in the quotes of researchers?

Satisfactory

As noted above, most of the “promise” of the breath test is based on hoped-for developments, not what was actually demonstrated in this trial. It is too early to say whether the ability to often distinguish between compounds in the breath of healthy people and those in the breath of people with established, often advanced, cancers means that it is at all likely that this test will someday be able to pick out important cancers in people in the general population.

Total Score: 5 of 10 Satisfactory

Comments

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.