Read Original Story

Post coverage of new lung cancer test misses on caveats, costs, and harms

New test could sharply reduce risky, costly lung cancer biopsies

Our Review Summary

X-ray doctorThis story describes results of studies reported in the New England Journal of Medicine about the value of a now commercially available genomic “classifier” test for lung cancer that has the potential to reduce unnecessary, invasive, and costly lung biopsies and surgeries. The new diagnostic test is one that relies on a proprietary RNA analysis of lung epithelial cells, and was developed with data from various procedures performed on 639 men and women. The story reports the test to be “97 percent accurate”; and it makes much of the fact that the new test will be marketed by a private company, and had to be developed over a long period of time with private venture capital and a company set up by the Boston University investigator after he struggled for years to find funding.

So far so good. But the 97% statistic comes with some qualifiers that aren’t mentioned, including the fact that this was a very carefully selected group of study patients that may not reflect the test’s performance in a broader population. And while it mentions the risks, costs, and deficiencies of needle biopsies and lung surgery to diagnose cancer, the article doesn’t mention the relatively high number of false positives revealed with use of the new test, or the cost of the new test ($3000-$4000). The absence of any independent commentators and the article’s omissions may leave readers with the impression that this test is a one-time slam-dunk diagnostic tool. In fact it’s highest value is in “re-classifying” many (mostly smokers) in an intermediate risk category to low risk, still necessitating regular CT scans and bronchoscopes.


Why This Matters

Although the article doesn’t mention cigarette smoking as the major risk factor in the persistently high rates of lung cancer and the need to screen for it, The prospect of having a less invasive and more precise means of finding cancers in those at risk is certainly big news for smokers. From a public health perspective, candidates for screening fall into low, intermediate, and high risk categories based on smoking history, genetics, and other occupational and environmental factors. Moreover, under terms of the Affordable Cancer Act, some 8 million Americans are newly eligible for low-dose CT scans for lung cancer screening and further testing with bronchoscopes, needle biopsies and operations. All of those procedures carry substantial risks and costs, so anything able to reduce unnecessary procedures and increase screening accuracy is important, even though it is an open question whether improved screening will find cancers early enough to raise the consistently low rate of cure with surgery, chemotherapy, and radiation. We wish the article had mentioned some of this background.

[Editor’s note: As pointed out by the story’s author in the comments, this review erroneously states that the story didn’t mention smoking as a major cause of lung cancer. The story did, in fact, include that information, and we’ve struck the erroneous passage from the review — see above under “Why This Matters.” We apologize for the error.] 


Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The article notes that traditional means of diagnosing suspected lung cancer are costly, often involving invasive procedures and resulting complications. That’s good. But it doesn’t cite any range of costs, information easily available from groups like the American Thoracic Society. More significantly, perhaps, is the lack of any estimated cost for the new genomic classifier test, which was apparently readily available because it was reported elsewhere. (A Reuters story, for one, had this info.)

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

Overall, the benefits of the new test appear to be potentially substantial, but the article should have quantified them with a bit more precision. For example, the “97 percent” accuracy rate used to describe the study results refers to the combined use of bronchoscopy and the new test — not the test by itself. Bronchoscopy alone is still accurate 75 percent of the time, and the test alone had an “area under the curve” of 0.74 – 0.78 — meaning that 74-78% of the time it could correctly classify whether or not a patient had cancer. It also would have made more sense, and brought more important context to readers, if the article had noted that the new test was able to reclassify a particular group of people at risk 91 percent of the time. This group is composed of those considered at intermediate risk. The ability to place them in a “watch and wait” or “low-risk” category is important because for some that will bring reassurance, but for others more anxiety and more tests.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The article notes that the additional maneuvering of the bronchoscope to collect cells for the genomic testing adds less than a minute to a bronchoscopy procedure. But it doesn’t make clear that bronchoscopy itself carries some risks (mainly of infections). In addition, a company news release reports that the test has a relatively low specificity of 47%, which means there will be many false positive results. The story doesn’t explore this beyond saying that there will still be biopsies. These false positives will result in unneeded surgical or needle biopsies in those who turn out not to have cancer. Any story about a new test – rather than dwelling on “accuracy” – should explain both the sensitivity and specificity of the test.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The article would have benefited from even a few details about the nature of the studies, the participants, and the way the genomic testing was done. For example, all of the participants were current or former smokers already undergoing bronchoscopy for suspected lung cancer. However, such patients represent a very small subset of those with abnormal CT scans for lung cancer. A repeat CT scan is by far the most common follow-up for people with an abnormal finding. Thus the results do not apply to most patients who are being scanned for lung cancer.

The researchers also excluded various groups of patients from their analysis. The “gold-standard” criterion for being cancer-free included a specific diagnosis of a benign condition or a stable/resolving imaging study. However, the study analyses excluded subjects who did not have a specific diagnosis or stable/resolving imaging study — which could lead to overestimating the performance of the new test. Five percent of patients fell into this category — as noted in the limitations section of the study. Furthermore, the study noted that specimens from 11% of subjects were inadequate for performing the genomic testing. We also need more clinical trial data (with longer follow up) before concluding that a negative test could safely “allow doctors to wait and watch a lesion” as the story says.

Does the story commit disease-mongering?


The article notes accurately that 160,000 Americans will likely die of lung cancer this year. If anything, the article doesn’t monger enough. The American Cancer Society estimates 221,000 new cases of lung cancer in 2015, most of them linked to tobacco smoking, and lung cancer is the biggest cancer killer in both men and women in the United States. Death rates remain very high.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The good news is that the study’s publication in a peer reviewed journal of high reputation suggests that the data have been vetted and the information is valid. That said, however, news articles that rely on journal articles and statements by commercial interests are greatly strengthened by comments from independent sources. This article clearly outlines the commercial interests of the primary investigator and developer of the genomic test. But readers might have learned more about the limitations of the study and the test if other experts had been interviewed.

Does the story compare the new approach with existing alternatives?


The article does a good job of describing current tests and what they can and cannot do. And it notes that although there are serious efforts to develop blood tests that can predict response to treatment and even early cases of lung cancer, these are likely years away from clinical use.

Does the story establish the availability of the treatment/test/product/procedure?


The article notes that a company bought the test and is “making it available” to hospitals nationwide.

Does the story establish the true novelty of the approach?


The article does an excellent job of describing the current methods of diagnosing lung cancer and how the genomic classifier differs.

Does the story appear to rely solely or largely on a news release?

Not Applicable

It’s unclear where the bulk of the article’s information came from beyond an interview with the test’s developer, a professor of computational biology and researcher at Boston University’s medical school. Veracyte, the company that bought the test, issued a release via PR Newswire and made researchers and company officials available to reporters in a conference call. If the story relied on those sources, it should have noted that was the case. Inclusion of an independent source would have erased any doubt in our minds as to the story’s reliance on any news release. But since we can’t tell either way where the information came from, we’ll rate this Not Applicable.

Total Score: 4 of 9 Satisfactory

Comments (2)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Lenny Bernstein

May 21, 2015 at 1:04 pm

Thanks for your thoughtful review. There’s one substantial error in the “Why This Matters” section.
It says: “Although the article doesn’t mention cigarette smoking as the major risk factor in the persistently high rates of lung cancer and the need to screen for it, the prospect of having a less invasive and more precise means of finding cancers in those at risk is certainly big news for smokers.”
In fact, I was very careful to point that out: “With the vast majority of lung cancer victims being smokers, the epithelial cells show changes that could be tracked once the technology became available, he said.”
I hope you’ll fix this, just for the record.


    Kevin Lomangino

    May 21, 2015 at 1:19 pm


    We’ve fixed the erroneous passage and added an editor’s note of explanation. We apologize for the mistake.

    Kevin Lomangino, Managing Editor