Health News Review

We’re told that researchers constructed a model – but there is no critical evaluation of that model.

Our Review Summary

HealthDay’s story didn’t make an attempt to critically evaluate the statistical modeling study as WebMD and TIME.com did.  This is a critical flaw in such a story.


Why This Matters

PSA screenings are one of the most talked about and written about facets of cancer care. Readers – especially men in 40 and older – can be easily confused by what seem to be conflicting studies about the benefits and harms of PSA tests. After reading three stories about the study – TIME.com, HealthDay, and WebMD – you might feel satisfied that all the relevant ground was covered, and that the most compelling evidence would be given the proper weight. But in every story, there were deficits.


Criteria

Not Satisfactory

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

We reviewed three stories that covered this new retrospective study on PSA screening. None of them discussed the costs involved in the PSA tests or the subsequent treatments.

Not Satisfactory

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story does accurately quantify the benefits of PSA screening as described in the study. It says, “In the 2008 data, 8,000 cases of prostate cancer were diagnosed after the malignancy had spread to other parts of the body. Using these cases as a base, the researchers constructed a model that used data of advanced cancer diagnosed in the 1980s and predicted how many cases of advanced cancer would have been diagnosed in 2008 if PSA testing was not done. Their model showed instead of 8,000 actual cases in 2008, about 25,000 cases would have been diagnosed.”

But there is no evaluation of the model itself.  This is difficult stuff for a journalist to do – but if you’re going to report on such studies, you must.  The TIME.com story at least took a stab at questioning the model. WebMD did better.

Satisfactory

Does the story adequately explain/quantify the harms of the intervention?

Satisfactory

The story puts some hard numbers to the potential side effects, with help from Dr. Michael LeFevre, co-vice chair of the U.S. Preventive Services Task Force (USPSTF).  The story says, “As a result of treatments for PSA-detected prostate cancer, one out of 1,000 men screened in the United States develops a blood clot in his legs or lungs, two will have a heart attack or stroke, and up to 40 are left impotent or with urinary incontinence, LeFevre said. ‘At best, only one of these 1,000 men who were screened avoids dying from prostate cancer for at least 10 years,’ LeFevre said. ‘In addition, about one in every 3,000 men screened dies as a result of surgery to treat cancer detected by the PSA test.’

Not Satisfactory

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story did describe the study’s design and explained that the findings were statistical estimates. But it would be easy for a reader to misunderstand and assume that the study is proof that lives actually were saved as a result of the PSA. The story should have raised some of the same concerns about the study’s limitations that were raised in the TIME.com and WebMD stories.

Satisfactory

Does the story commit disease-mongering?

Satisfactory

HealthDay framed the discussion about PSA tests nicely by saying, “Many cases of prostate cancer are not life-threatening, which is why testing is controversial.” Contrast that with how TIME.com described the debate about whether to have a PSA test: “It depends on how much you value certain quality-of-life measures versus how much you value having any life at all.”

Satisfactory

Does the story use independent sources and identify conflicts of interest?

Satisfactory

Expert perspectives from Dr. Otis Brawley of the American Cancer Society and from Dr. Michael LeFevre of the USPSTF helped this story.

Satisfactory

Does the story compare the new approach with existing alternatives?

Satisfactory

The alternative to PSA screening is the choice to not be screened – and that is at the core of the story.

Satisfactory

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The story made it clear that PSA screening is widely available.

Not Applicable

Does the story establish the true novelty of the approach?

Not Applicable

There were no claims of novelty in the story.

Satisfactory

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story did not rely on a press release.

Total Score: 6 of 9 Satisfactory


We Welcome Comments

But please note: We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, profanity or any from anyone who doesn't list what appears to be an actual email address. We will also end any thread of repetitive comments. We don't give medical advice so we won't respond to questions asking for it. Please see more on our comments policy.