News releases can lead media like sheep – hiding key problems

Reviewing news releases about health care is more than an academic exercise.

It matters because news stories that reach the public are often directly influenced by the content of news releases. That’s what news releases are intended to do. It is, sadly, sometimes like sheep following each other.

Today we published our first systematic, criteria-driven review of a health care news release. We’ve been doing this behind the scenes for the past few months while our new website was being worked on.

The release we reviewed came from the University of Wisconsin Health Marketing and Public Affairs department.

Our reviewers – there were four of them – had problems with the review, writing:

“Misleading to the point of deception, this news release ignores all primary and secondary analyses of the study it ostensibly reports on — which were uniformly negative and showed no benefit — and instead treats a positive subgroup analysis as if it were the main finding of the study.”

But the story was picked up by news organizations as far away as the Tampa Bay Times – http://www.tampabay.com/news/health/medicine/lupron-depot-slows-alzheimers-study-shows-but-it-may-not-matter/2222480 – to many websites such as:

• Alzheimer’s News Today

• Alzheimers.Net

• 1HeartCares.com

• FormularyWatch

Why does this matter? Our reviewers addressed that:

“Since the drugs used in combination in this study (Lupron and acetylcholinesterase inhibitors) are already on the market, it’s conceivable that desperate family members and caregivers of Alzheimer’s patients will demand treatment based on these hyped results. (A broadcast television news story even encourages patient family members to “ask their doctors to consider putting them on the drug.”) The benefits touted by the release were seen only in a smaller subgroup of patients — not the main study — and subgroups need to be treated very carefully for a variety of reasons.”

WKOW 27: Madison, WI Breaking News, Weather and Sports

But none of these outlets explained anything about the fact that the results being touted came from a subgroup analysis – an issue which, in general, is important enough to drive some authors to write: “While it is general recognized that subgroup analyses can produce spurious results, the extent of the problem is almost certainly under-estimated.”

Other researchers have written in the past: “Treatment effects in subgroups might offer clinicians more insight into treating individual patients. However, if sometimes they are informative, subgroup analyses are more frequently misleading. Subgroup analyses have been characterized as ‘a scientific challenge and a methodological trap,’ and many authors have cautioned against overinterpreting them.”

Indeed, it’s not difficult at all to find warnings about the limitations of drawing conclusions from subgroup analyses. There are entire web pages dedicated to cautions about subgroup analyses.

But you wouldn’t know it from the news stories and web articles that flowed from the news release.

Because the news release didn’t even mention subgroup analysis.

And that’s why we’re going to be systematically critiquing health care news releases from now on – to try to help people learn how to evaluate claims and to evaluate evidence.