This story describes results of studies reported in the New England Journal of Medicine about the value of a now commercially available genomic “classifier” test for lung cancer that has the potential to reduce unnecessary, invasive, and costly lung biopsies and surgeries. The new diagnostic test is one that relies on a proprietary RNA analysis of lung epithelial cells, and was developed with data from various procedures performed on 639 men and women. The story reports the test to be “97 percent accurate”; and it makes much of the fact that the new test will be marketed by a private company, and had to be developed over a long period of time with private venture capital and a company set up by the Boston University investigator after he struggled for years to find funding.
So far so good. But the 97% statistic comes with some qualifiers that aren’t mentioned, including the fact that this was a very carefully selected group of study patients that may not reflect the test’s performance in a broader population. And while it mentions the risks, costs, and deficiencies of needle biopsies and lung surgery to diagnose cancer, the article doesn’t mention the relatively high number of false positives revealed with use of the new test, or the cost of the new test ($3000-$4000). The absence of any independent commentators and the article’s omissions may leave readers with the impression that this test is a one-time slam-dunk diagnostic tool. In fact it’s highest value is in “re-classifying” many (mostly smokers) in an intermediate risk category to low risk, still necessitating regular CT scans and bronchoscopes.
Although the article doesn’t mention cigarette smoking as the major risk factor in the persistently high rates of lung cancer and the need to screen for it, The prospect of having a less invasive and more precise means of finding cancers in those at risk is certainly big news for smokers. From a public health perspective, candidates for screening fall into low, intermediate, and high risk categories based on smoking history, genetics, and other occupational and environmental factors. Moreover, under terms of the Affordable Cancer Act, some 8 million Americans are newly eligible for low-dose CT scans for lung cancer screening and further testing with bronchoscopes, needle biopsies and operations. All of those procedures carry substantial risks and costs, so anything able to reduce unnecessary procedures and increase screening accuracy is important, even though it is an open question whether improved screening will find cancers early enough to raise the consistently low rate of cure with surgery, chemotherapy, and radiation. We wish the article had mentioned some of this background.
[Editor’s note: As pointed out by the story’s author in the comments, this review erroneously states that the story didn’t mention smoking as a major cause of lung cancer. The story did, in fact, include that information, and we’ve struck the erroneous passage from the review — see above under “Why This Matters.” We apologize for the error.]
The article notes that traditional means of diagnosing suspected lung cancer are costly, often involving invasive procedures and resulting complications. That’s good. But it doesn’t cite any range of costs, information easily available from groups like the American Thoracic Society. More significantly, perhaps, is the lack of any estimated cost for the new genomic classifier test, which was apparently readily available because it was reported elsewhere. (A Reuters story, for one, had this info.)
Overall, the benefits of the new test appear to be potentially substantial, but the article should have quantified them with a bit more precision. For example, the “97 percent” accuracy rate used to describe the study results refers to the combined use of bronchoscopy and the new test — not the test by itself. Bronchoscopy alone is still accurate 75 percent of the time, and the test alone had an “area under the curve” of 0.74 – 0.78 — meaning that 74-78% of the time it could correctly classify whether or not a patient had cancer. It also would have made more sense, and brought more important context to readers, if the article had noted that the new test was able to reclassify a particular group of people at risk 91 percent of the time. This group is composed of those considered at intermediate risk. The ability to place them in a “watch and wait” or “low-risk” category is important because for some that will bring reassurance, but for others more anxiety and more tests.
The article notes that the additional maneuvering of the bronchoscope to collect cells for the genomic testing adds less than a minute to a bronchoscopy procedure. But it doesn’t make clear that bronchoscopy itself carries some risks (mainly of infections). In addition, a company news release reports that the test has a relatively low specificity of 47%, which means there will be many false positive results. The story doesn’t explore this beyond saying that there will still be biopsies. These false positives will result in unneeded surgical or needle biopsies in those who turn out not to have cancer. Any story about a new test – rather than dwelling on “accuracy” – should explain both the sensitivity and specificity of the test.
The article would have benefited from even a few details about the nature of the studies, the participants, and the way the genomic testing was done. For example, all of the participants were current or former smokers already undergoing bronchoscopy for suspected lung cancer. However, such patients represent a very small subset of those with abnormal CT scans for lung cancer. A repeat CT scan is by far the most common follow-up for people with an abnormal finding. Thus the results do not apply to most patients who are being scanned for lung cancer.
The researchers also excluded various groups of patients from their analysis. The “gold-standard” criterion for being cancer-free included a specific diagnosis of a benign condition or a stable/resolving imaging study. However, the study analyses excluded subjects who did not have a specific diagnosis or stable/resolving imaging study — which could lead to overestimating the performance of the new test. Five percent of patients fell into this category — as noted in the limitations section of the study. Furthermore, the study noted that specimens from 11% of subjects were inadequate for performing the genomic testing. We also need more clinical trial data (with longer follow up) before concluding that a negative test could safely “allow doctors to wait and watch a lesion” as the story says.
The article notes accurately that 160,000 Americans will likely die of lung cancer this year. If anything, the article doesn’t monger enough. The American Cancer Society estimates 221,000 new cases of lung cancer in 2015, most of them linked to tobacco smoking, and lung cancer is the biggest cancer killer in both men and women in the United States. Death rates remain very high.
The good news is that the study’s publication in a peer reviewed journal of high reputation suggests that the data have been vetted and the information is valid. That said, however, news articles that rely on journal articles and statements by commercial interests are greatly strengthened by comments from independent sources. This article clearly outlines the commercial interests of the primary investigator and developer of the genomic test. But readers might have learned more about the limitations of the study and the test if other experts had been interviewed.
The article does a good job of describing current tests and what they can and cannot do. And it notes that although there are serious efforts to develop blood tests that can predict response to treatment and even early cases of lung cancer, these are likely years away from clinical use.
The article notes that a company bought the test and is “making it available” to hospitals nationwide.
The article does an excellent job of describing the current methods of diagnosing lung cancer and how the genomic classifier differs.
It’s unclear where the bulk of the article’s information came from beyond an interview with the test’s developer, a professor of computational biology and researcher at Boston University’s medical school. Veracyte, the company that bought the test, issued a release via PR Newswire and made researchers and company officials available to reporters in a conference call. If the story relied on those sources, it should have noted that was the case. Inclusion of an independent source would have erased any doubt in our minds as to the story’s reliance on any news release. But since we can’t tell either way where the information came from, we’ll rate this Not Applicable.
Systematic, Criteria-Driven
New ep of @hilarious_is with @garyschwitzer of @HealthNewsRevu - he's the Michael Corleone of myth-busting in health and medicine news!
Just
Press
Play
#podcast #journalism #FactCheck #science #medicine
https://www.patreon.com/posts/42537673
@garyschwitzer has been a trailblazer in protecting us from bamboozled health reporting. Thrilled to get his endorsement here for #Malcharist, which fictionalizes problems he first helped us see. https://twitter.com/garyschwitzer/status/1310190671555375109
Comments (2)
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Lenny Bernstein
May 21, 2015 at 1:04 pmThanks for your thoughtful review. There’s one substantial error in the “Why This Matters” section.
It says: “Although the article doesn’t mention cigarette smoking as the major risk factor in the persistently high rates of lung cancer and the need to screen for it, the prospect of having a less invasive and more precise means of finding cancers in those at risk is certainly big news for smokers.”
In fact, I was very careful to point that out: “With the vast majority of lung cancer victims being smokers, the epithelial cells show changes that could be tracked once the technology became available, he said.”
I hope you’ll fix this, just for the record.
Best,
Lenny
Kevin Lomangino
May 21, 2015 at 1:19 pmLenny,
We’ve fixed the erroneous passage and added an editor’s note of explanation. We apologize for the mistake.
Kevin Lomangino, Managing Editor
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like