The following is a guest post by Dr. Richard Hoffman, one of our longtime story reviewers and blog contributors. He recently became the Director of the Division of General Internal Medicine for the University of Iowa Carver College of Medicine/Iowa City VA Medical Center.
Last week the American College of Physicians published position papers on achieving high value from cancer screening (see here and here). Ideally, “high-value strategies” deliver a large health benefit relative to harms and costs. Prostate cancer screening, the papers implied, has not been meeting this standard.
One major problem with prostate cancer screening is that the prostate-specific antigen (PSA) test is not very accurate. Only about 30% of men with an abnormal PSA result will have prostate cancer—as determined by a prostate biopsy. This low predictive value has led investigators to evaluate strategies for reducing unnecessary biopsies. While these strategies theoretically could reduce false positives, studies often failed to fully address the problem of overdiagnosis—finding slow-growing, low-grade tumors that would never cause problems during a man’s lifetime. Overdiagnosis and resulting overtreatment undermine the value of prostate cancer screening.
A study recently presented at the American Urological Association’s annual meeting addressed the overdiagnosis issue. Investigators showed that a novel gene signature test performed on a urine specimen could better predict which men with an intermediate-level elevation of PSA (2 to 10 ng/mL) would have a high-grade prostate cancer (Gleason scores of 7 or greater) on biopsy. Based on the study results, investigators postulated that the new test could reduce biopsies by 27%, while missing few of the highest-grade tumors.
These findings are interesting but preliminary. The study was not designed to address whether using this test for clinical decision making actually improved patient outcomes—fewer biopsies and at least similar cancer control—compared with other ways of assessing risk.
Unfortunately, New York-Presbyterian Hospital, home of the lead investigator, issued a news release that was notable for misinforming readers about the study. The release was titled, “Urine test detects prostate cancer with 92 percent accuracy, according to study.” The AUA abstract presented enough data for me to construct a 2 x 2 table.
High-grade cancer | Low-grade cancer/benign | Totals | |
Test positive | 137 | 244 | 381 |
Test negative | 12 | 126 | 138 |
Totals | 149 | 370 | 519 |
The only “92%” number calculable from the data is the sensitivity—the proportion of high-grade cancers that can be detected with the new test (137/149). But the news release talks about “accuracy,” which as defined in the epidemiology literature is a fairly useless concept. It refers to the proportion of all tested subjects who are correctly classified—the true positives + the true negatives. In this case, the “accuracy” of the new test is only (137+126)/519 = 51%. Although not mentioned in the press release, the abstract reported the area under the ROC curve (AUROC)—which is the best overall measure of how well a test distinguishes between people with disease and those without. The AUROC for the gene test was 0.72 (a value of 0.5 corresponds to a worthless test while 1.0 corresponds to a test with perfect accuracy)–indicating fair to good performance.
I believe the most important implications of the study are that if men with a negative gene test were not biopsied, then overall 27% (138/519) fewer men would undergo biopsy. These men would avoid the anxiety and risk of complications — including potentially serious infections — that accompany this procedure. Furthermore, among these men avoiding biopsy, 91% (126/138; the negative predictive value) would not have a high-grade cancer—but 9% would. Men who are making biopsy decisions based on the new test would need to understand this tradeoff.
But the release does not frame the results in a way that would promote such informed decision-making. In fact, some of the quotations in the news release were downright perplexing. For example:
What could this news release have done differently to more effectively communicate the value of this research? In my view, highlighting a meaningless “accuracy” statistic and presenting misleading quotes about the new test “replac[ing] traditional surgical biopsies” and being “more accurate than traditional biopsies” just obscure the actual study findings. The release should have focused on the test’s potential to avoid unnecessary biopsies while still identifying almost all of the more aggressive cancers. A tumor marker that could potentially prevent unnecessary biopsies and reduce overdiagnosis would improve the value of prostate cancer—this is the important message.
[Editor’s note: Contributor Trudy Lieberman recently wrote about her concerns with aspects of New York Presbyterian’s consumer advertising in her post, “Patient anecdotes in NY-Presbyterian hospital ads mask mediocre quality ratings.”]
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Comments are closed.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like