Headline vs. study: A battle where readers often lose

Posted By

Kevin Lomangino is the managing editor of HealthNewsReview.org. He tweets as @Klomangino.

headline-vs-study-3The quest for balance in a health news story can fail before the first sentence if the headline isn’t appropriately calibrated. With that in mind, I looked at news stories and releases that we reviewed over the past month and compared the headline message with that of the study on which the news is based.

About a third of news story headlines and a quarter of news release headlines either misstated the results or went beyond what the research could support. It’s an admittedly small sample of our recent work, but I don’t think the results are out of line with what we see month in and month out. I hope the findings shine a light on the ways that headline messages mislead – and how headline writers can do better.

News story report card: 5 of 16 (31%) headlines overstate study evidence

 

hookworms for asthma

Credit: Korean Academy of Medical Sciences

Headline: Bloodsucking parasitic hookworms could help make millions of people healthier

Study: Experiments in mice — not people — showed benefits on asthma symptoms.

Our review: “…the story provides no sense of how challenging it will be to translate these results to humans, a very important caveat that should have been discussed.”


doctor-with-childHeadline: Testing cholesterol in toddlers, even younger? Study says it could help

Study: The study found that a genetic high cholesterol condition was more common in babies than previously thought. It didn’t address whether identifying the condition at a younger age ‘could help’ — an entirely different issue.

Our review: “…the story’s framing takes the stance that finding the risk factors for this disease very early on in life leads to an overall benefit. But the study provides no evidence of that, and parents need to know this.”


Pensive Beautiful little girlHeadlineStudy offers potential breakthrough in care of children with autism

Study: The study found no statistically significant difference between treatment and control groups for the main outcome — a fact never mentioned in the story.

Our review: “…saying that researchers have found a ‘potential breakthrough’ for results that might have been due to chance seems misleading..”


mouse researchHeadlineHow Omega 3 Fats May Improve Fertility

Study: Mice — not people — bred to have higher ratios of omega 3 fatty acids seemed to produce more “precursors to egg cells” and also produced higher quality eggs, meaning they would be more likely to have eggs that would be fertilized and develop into baby mice.

Our review: “The story provides no sense of what’s involved, or just how challenging it will be, to translate these results to humans. So why report it now when there’s no evidence? We don’t know.”


lung cancer illustration

Headline: ‘New day’ in lung cancer as Merck drug shines, works with chemo

Study: The drug delayed progression of often-fatal lung cancers for an additional 4 months compared with chemo in patients whose tumors expressed a certain biomarker. The duration of benefit is never mentioned in the story, nor is the number of lung cancer patients who might be candidates for treatment.

Our review:  “Its ‘new day’ headline and overall optimistic tone may mislead some people into thinking the drug offers lengthy survival (or even a cure) for anyone with this type of often-lethal cancer.”

News release report card: 4 of 17 (24%) headlines overstate evidence

 

depressed, sleepingHeadline: Easing Labor Pain May Help Reduce Postpartum Depression In Some Women, Early Research Suggests

Study: Epidural use was associated with reduced risk of postpartum depression in this observational study that cannot show cause and effect. Suggesting that pain relief “may help reduce postpartum depression” is not justified by the evidence.

Our review: “The tone of the news release and the abstract (both by the American Society of Anesthesiologists) suggests that epidural anesthesia is helpful in preventing postpartum depression, but that assertion remains far from proven.”


whole grains, fiberHeadline: Cleveland Clinic study finds whole grain diet reduces cardiovascular disease risk 

Study: The study documented a reduction in blood pressure when participants ate more whole grains for 8 weeks. It did not document that this small, short-term reduction in blood pressure had any effect on cardiovascular disease risk or outcomes — a fact never clarified in the release.

Our review: “…the lowered blood pressure was a surrogate outcome and not a primary endpoint, i.e. reduced risk of cardiovascular disease. It would take a much larger outcomes study to demonstrate that.”


Pill for Alzheimer'sHeadline: Phase 3 analyses in Alzheimer’s show clinical benefit of tramiprosate in APOE4 carriers

Study: The published study istates that the research — based on a 2,000 patient cohort — did “not reach its primary objectives.” This is never acknowledged in the release. The headline is based on a smaller subgroup analysis.

Our review:  “…this sub-group analysis cannot lead one to draw any conclusions about whether the drug “worked,” given that the overall study was negative.”


IVHeadline: ‘Game-changing’ immunotherapy doubles head and neck cancer survival

Study: Median overall survival was 7.5 months in the treatment group and 5.1 months in the control group. The “game-changing” result amounts to a 2-month increase in survival time.

Our review: “That sort of vague, and potentially misleading, language has little place in a news release like this one.” .

Note: the inspiration for this analysis came from Dr. David Allison’s “Obesity and Energetics Offerings,” a weekly newsletter sent from the University of Alabama, Birmingham. 

You might also like

Comments (4)

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.

Florence d'Eon

November 15, 2016 at 9:29 am

The results of the Ottawa Hospital study (Dr. Mark Freedman and Dr. Harold Atkins) were called ‘impressive’ and ‘Like a miracle’. However, The Lancet editorial notes the trial was not controlled by a placebo group a “most important limitation”. Study had only 24 patients (one died). A positive spin on this study was copied by news media all over the world creating a lot of hype without a critical review of the study stating limitations. Hype before the science.

Sally James (@jamesian)

November 15, 2016 at 12:10 pm

Great story. I’m now imagining “mouse” emoji that publishers have to place in the headline next to research only conducted in mice. Would not take up much space.

Pat Bowne

November 16, 2016 at 5:36 am

Good for you for looking into this. I tweet for a teachers’ organization, and have often comented on sites like this that I needed to know whether the results reported were about humans or mice early in the article – often to see comments like ‘just assume it’s in mice’ posted in reply. Since all of us who are in science teaching probably give our students criteria for identifying credible sources and teach them how to write accurate captions for their graphs, etc., there should be lots of options out there for creating a headline-writing rubric for the field.

Christine Rosenbloom

November 18, 2016 at 2:17 pm

Love this article; sad, but true! And, love the idea of a “mouse emoji.” Great idea Sally James!