Michael Joyce is a writer-producer with HealthNewsReview.org and tweets as @mlmjoyce
This edition of Headline vs. Study focuses on two recurrent problems we see in both news stories and news releases.
First, the prevailing assumptions that either new technology (like using stem cells for a common knee ailment), or more technology (like combining two heart scan techniques) are automatically superior to the existing approach. Although that sometimes may be the case, it’s only the case when the supporting data has been rigorously tested and re-tested.
Second, beware of single case studies or unpublished studies sponsored by industry. There’s nothing wrong with writing about such research but the limitations of these types of studies should be disclosed to readers and the headlines should not promise more than the study results can support.
Here’s a look at some headlines going back over the past month, the promises they make, and why they don’t deliver on that promise.
Headline: Cardiac hybrid imaging an effective tool for predicting heart attacks
Study: The research looks at using a combination of two heart imaging techniques: coronary computed tomography angiography (CCTA) and single photon emission tomography (SPECT) — to produce what the release says is a more accurate picture of cardiac blood flow.
Our review: “There’s a clear mismatch between the overstated headline and the more accurate language featured in the release’s conclusion. The headline describes the hybrid approach as an ‘effective tool’ for evaluating coronary artery disease; [however] the last paragraph acknowledges that the approach hasn’t been tested in a clinical trial and that we don’t know if it actually has a positive impact on patient outcomes.”
Headline: Stem cells show promise for repairing torn meniscus
Study: A single case study … NOT a clinical trial (or even a case series). Without any control group, it’s impossible to know whether any improvement in symptoms is attributable to stem cells or some other factor.
Our review: “The news release doesn’t give any data on how large/small are the potential benefits. It only offers general statements such as, ‘the patient reported a reduction in pain and an improvement in knee function’.”
The release highlights benefits while not mentioning any adverse effects. This would make it difficult for readers to draw any conclusions about the safety of the procedure, especially given it’s a single case report with a short follow-up of just 12 months.
Headline: Sugar improves memory in over-60s, helping them work smarter
Study: A small, short-term trial that suggests glucose (a form of sugar) enhances performance on memory tests in older adults.
Our review: It’s not clear whether a short-term boost on a lab-based memory test can help people “work smarter” and the release doesn’t clarify.
Our reviewers said “the release touts benefits in performance, motivation, and mood while taking these tests, but fails to include any data. Nor does it mention how these outcomes were defined or measured.”
Neither potential harms of glucose supplementation, or the limitations of the study, are mentioned in the release.
Headline: A flu drug — shown to reduce the duration of symptoms — could upend treatment in U.S.
Study: There’s no published study; rather, the news release is prompted by an FDA announcement that the Genentech single-dose flu drug (baloxavir marboxil) has been granted a priority review.
Our review: The claim that this drug will reduce the duration of flu symptoms is not backed up by the evidence. Our reviewers noted: “Similar efficacy results were seen between baloxavir marboxil and [Tamiflu] in relation to duration of symptoms and fever reduction.” In other words, on the symptoms front, both drugs were the same.
Also, it’s premature to claim in your headline that a drug “could upend treatment” when the study has yet to be peer-reviewed and published.
Headline: Maintaining healthy vision may help keep brain in shape, too
Study: It’s been established that clearness of vision and cognitive function seem to rise and fall together in older adults. But does one cause the other?
“This study introduced time into the analysis, exploring the relationship between clarity of vision at time 1 and cognitive function at time 2, and vice versa, repeatedly over an 8-year period.” It does find a relationship but, given the large number of participants in the study (more than 2,500), it’s a modest one, and its impact — if any — would still need to be tested in a clinical trial to be proven conclusively.
Our review: “The story describes a relationship between vision and cognition, but we never get a feel for how strong that link might be … alas, that does not deter either sources in the story or the headline writer, who opt into causal statements that are far stronger than the data warrant.”
You can find more from our Headline vs. Study series HERE
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Comments are closed.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like