Headline vs. study: Promises for your heart, brain, and the common flu

This edition of Headline vs. Study focuses on two recurrent problems we see in both news stories and news releases.

First, the prevailing assumptions that either new technology (like using stem cells for a common knee ailment), or more technology (like combining two heart scan techniques) are automatically superior to the existing approach. Although that sometimes may be the case, it’s only the case when the supporting data has been rigorously tested and re-tested.

Second, beware of single case studies or unpublished studies sponsored by industry. There’s nothing wrong with writing about such research but the limitations of these types of studies should be disclosed to readers and the headlines should not promise more than the study results can support.

Here’s a look at some headlines going back over the past month, the promises they make, and why they don’t deliver on that promise.

News RELEASE report card: 7 of 10 (70%) headlines overstate evidence (we list 3 examples below)

Headline: Cardiac hybrid imaging an effective tool for predicting heart attacks

Study: The research looks at using a combination of two heart imaging techniques: coronary computed tomography angiography (CCTA) and single photon emission tomography (SPECT) — to produce what the release says is a more accurate picture of cardiac blood flow.

Our review: “There’s a clear mismatch between the overstated headline and the more accurate language featured in the release’s conclusion. The headline describes the hybrid approach as an ‘effective tool’ for evaluating coronary artery disease; [however] the last paragraph acknowledges that the approach hasn’t been tested in a clinical trial and that we don’t know if it actually has a positive impact on patient outcomes.”

Headline: Stem cells show promise for repairing torn meniscus

Study: A single case study … NOT a clinical trial (or even a case series). Without any control group, it’s impossible to know whether any improvement in symptoms is attributable to stem cells or some other factor.

Our review: “The news release doesn’t give any data on how large/small are the potential benefits. It only offers general statements such as, ‘the patient reported a reduction in pain and an improvement in knee function’.”

The release highlights benefits while not mentioning any adverse effects. This would make it difficult for readers to draw any conclusions about the safety of the procedure, especially given it’s a single case report with a short follow-up of just 12 months.

Headline: Sugar improves memory in over-60s, helping them work smarter

Study:  A small, short-term trial that suggests glucose (a form of sugar) enhances performance on memory tests in older adults.

Our review: It’s not clear whether a short-term boost on a lab-based memory test can help people “work smarter” and the release doesn’t clarify.

Our reviewers said “the release touts benefits in performance, motivation, and mood while taking these tests, but fails to include any data. Nor does it mention how these outcomes were defined or measured.”

Neither potential harms of glucose supplementation, or the limitations of the study, are mentioned in the release.

News STORY report card: 3 of 10 (30%) headlines overstate study evidence (we list 2 examples below)

Headline: A flu drug — shown to reduce the duration of symptoms — could upend treatment in U.S.

Study: There’s no published study; rather, the news release is prompted by an FDA announcement that the Genentech single-dose flu drug (baloxavir marboxil) has been granted a priority review.

Our review: The claim that this drug will reduce the duration of flu symptoms is not backed up by the evidence. Our reviewers noted: “Similar efficacy results were seen between baloxavir marboxil and [Tamiflu] in relation to duration of symptoms and fever reduction.” In other words, on the symptoms front, both drugs were the same.

Also, it’s premature to claim in your headline that a drug “could upend treatment” when the study has yet to be peer-reviewed and published.

Headline: Maintaining healthy vision may help keep brain in shape, too

Study: It’s been established that clearness of vision and cognitive function seem to rise and fall together in older adults. But does one cause the other?

“This study introduced time into the analysis, exploring the relationship between clarity of vision at time 1 and cognitive function at time 2, and vice versa, repeatedly over an 8-year period.”  It does find a relationship but, given the large number of participants in the study (more than 2,500), it’s a modest one, and its impact — if any — would still need to be tested in a clinical trial to be proven conclusively.

Our review:  “The story describes a relationship between vision and cognition, but we never get a feel for how strong that link might be … alas, that does not deter either sources in the story or the headline writer, who opt into causal statements that are far stronger than the data warrant.”

You can find more from our Headline vs. Study series HERE