This news release is about a blood test intended to more specifically define a man’s likelihood of having a high-grade prostate cancer tumor. It alerts readers several times to the preliminary nature of the small “proof of concept” study. However, highlighting a “94% accuracy” in the headline and lead paragraph is unjustified. The release does not mention cost, nor does it adequately discuss potential benefits or harms. The release implies that this test offers something unique, but it fails to tell readers that other blood tests are much farther along in the research process.
“Preliminary,” “proof of concept” and references to the need to confirm and expand the findings of a study all point out how much more work needs to be done in order to find out if this experimental blood test might offer anything of value. Putting a number on the “accuracy” of the test, especially without defining or explaining details of the assessment, implies a level of certainty that does not exist.
It is well known that the PSA test for prostate cancer screening isn’t very reliable. There are several efforts underway to develop improved tests. The biggest problem is that the PSA test isn’t very specific — meaning that a man with an abnormal test result may not have cancer. That is termed a “false positive” test. Even among those with an abnormal PSA test who are diagnosed with cancer, many of these cancers (some experts suggest up to 40%) may be so small or otherwise innocuous that they really should be left alone.
Can a new test be developed that identifies cancers that need to be treated more accurately than the currently available tests? As presented, a reader may surmise that this new test looks promising, but for a number of reasons it may be “too good to be true.” First, we don’t know very much about the men studied, but often patients in such initial studies are more likely to have prostate cancers that need treatment. When later the test is applied to a larger, more diverse population, the test may not perform as well. Moreover, the definition of what is a cancer that requires treatment is not clearly stated in this release. We know that the referenced Gleason score which is based upon the biopsy specimen isn’t perfect either and we’re not sure what criteria was used in this study. Finally, any new test that cuts down on false positive results is likely to lead to missing some cancers that really do need treatment. These are termed false negative test results.
Although this blood test is still experimental, the company must have some idea about how much it thinks it will charge if it ever gets into clinical use. Another way the release could have addressed cost, absent an estimated cost for this test, would be to comment on the cost of a PSA, other similar blood tests this company has developed for other conditions, or even the cost of a biopsy that in theory could be avoided.
It’s impossible to judge the potential benefits because basic information is missing. For example, the release does not define what it means by “high-grade” prostate cancer. It says it used the Gleason Score, but not what cutoff the company used in this analysis. Did they consider a Gleason score of 7 to be high-grade? Without such specifics, it is impossible to interpret the meaning of “94% accuracy.”
The release states:
“At 88% specificity, the Volition panel of five assays (including PSA) identified 94% of high-grade prostate cancers that require treatment (as defined by Gleason Score1). This compares with just 33% identified by Prostate-Specific Antigen (PSA) alone.”
There are problems with this data on several levels. First, they refer to the 94% as accuracy. But the description implies something else. The 94% refers to what sounds like the sensitivity of the test, not its accuracy. Sensitivity is the number of true cancers (a gold standard defined based upon the biopsy result showing a “high-grade” cancer) picked up by the test. So the 94% implies sensitivity or that 6% of patients with true cancers were missed by this new test. What is confusing is that this is compared to a 33% accuracy of the PSA test. And here also the new test is stated to have a specificity of 88%. So we’re left wondering if the 33% refers to the PSA’s specificity, and not the accuracy. If it’s not clear to us, it may not be clear to lay readers either.
For more explanation on sensitivity and specificity see our toolkit article on the topic.
There is no discussion of harms. How might the false positive rate compare with current practice? Even if, as the company hopes, this blood test might reduce the total number of men sent to have biopsies who turn out to have low-grade tumors, could this test send some men to biopsy who would not have been subjected to it based on current practice?
On the other hand, what is the false negative risk; that is, the chance that identification of a high-grade tumor might be delayed because a man was not referred for a biopsy based on the results of this blood test?
The quality of the evidence is insufficient. There is inadequate description of who these patients are. Many times by identifying and focusing on higher risk patients in a trial, the test will look better than when applied to a broader range of patients in routine practice.
We give the release some credit for calling attention to the preliminary nature of this research. However, since this is a small study and preliminary, highlighting a “94% accuracy” in the headline and lead paragraph overstates the evidence… especially since no information is provided about the range of the confidence interval around that number. There is really no way to say how accurate this test might be, based on the information in this release and the description of the study, so it would have been better to just leave out the accuracy claim.
It’s nice to see a prostate cancer release that notes, “most men with elevated PSA levels referred for prostate biopsy either have no cancer or have low-grade cancer which needs monitoring but not treatment.”
Although it is clear that this study is funded by a company hoping to market a new blood test and that most of the quotes come from company employees, the release should have explained the relationship between the company and the principal investigator. Did the company just provide funding for the study? Were there direct payments to the researchers, including consulting or lecturing fees outside of the study budget? What was the role of the company in designing the study and reporting the results?
The release directly compares this experimental blood test to current practice employing the PSA blood test, but there aren’t enough details about how the comparison was made for readers to get a sense of whether the claim of superiority has a reasonable basis. There is no justification for stating it is 94% accurate compared to PSA testing being 33% accurate at identifying high-grade tumors at this preliminary stage.
It is well known that the PSA is not a very accurate screening test. There are other attempts to improve upon it., including using MRIs to determine whether someone has a lesion that corresponds to the elevated PSA or not.
The release makes clear that this blood test is experimental and one can safely assume it is not yet available.
Based on what is provided in the news release (we could not find any published or unpublished studies) this appears to be a new approach to screening for prostate cancer.
The release highlights the test’s “94% accuracy” in the headline and text but only after reading the entire release does it become clear that this is very preliminary work, and the claim is unjustified.