Reviewing news releases about health care is more than an academic exercise.
It matters because news stories that reach the public are often directly influenced by the content of news releases. That’s what news releases are intended to do. It is, sadly, sometimes like sheep following each other.
Today we published our first systematic, criteria-driven review of a health care news release. We’ve been doing this behind the scenes for the past few months while our new website was being worked on.
The release we reviewed came from the University of Wisconsin Health Marketing and Public Affairs department.
Our reviewers – there were four of them – had problems with the review, writing:
“Misleading to the point of deception, this news release ignores all primary and secondary analyses of the study it ostensibly reports on — which were uniformly negative and showed no benefit — and instead treats a positive subgroup analysis as if it were the main finding of the study.”
But the story was picked up by news organizations as far away as the Tampa Bay Times – http://www.tampabay.com/news/health/medicine/lupron-depot-slows-alzheimers-study-shows-but-it-may-not-matter/2222480 – to many websites such as:
Why does this matter? Our reviewers addressed that:
“Since the drugs used in combination in this study (Lupron and acetylcholinesterase inhibitors) are already on the market, it’s conceivable that desperate family members and caregivers of Alzheimer’s patients will demand treatment based on these hyped results. (A broadcast television news story even encourages patient family members to “ask their doctors to consider putting them on the drug.”) The benefits touted by the release were seen only in a smaller subgroup of patients — not the main study — and subgroups need to be treated very carefully for a variety of reasons.”
WKOW 27: Madison, WI Breaking News, Weather and Sports
But none of these outlets explained anything about the fact that the results being touted came from a subgroup analysis – an issue which, in general, is important enough to drive some authors to write: “While it is general recognized that subgroup analyses can produce spurious results, the extent of the problem is almost certainly under-estimated.”
Other researchers have written in the past: “Treatment effects in subgroups might offer clinicians more insight into treating individual patients. However, if sometimes they are informative, subgroup analyses are more frequently misleading. Subgroup analyses have been characterized as ‘a scientific challenge and a methodological trap,’ and many authors have cautioned against overinterpreting them.”
Indeed, it’s not difficult at all to find warnings about the limitations of drawing conclusions from subgroup analyses. There are entire web pages dedicated to cautions about subgroup analyses.
But you wouldn’t know it from the news stories and web articles that flowed from the news release.
Because the news release didn’t even mention subgroup analysis.
And that’s why we’re going to be systematically critiquing health care news releases from now on – to try to help people learn how to evaluate claims and to evaluate evidence.
Comments
We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.
We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.
Comments are closed.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like