Headline vs. study: A battle where readers often lose

The quest for balance in a health news story can fail before the first sentence if the headline isn’t appropriately calibrated. With that in mind, I looked at news stories and releases that we reviewed over the past month and compared the headline message with that of the study on which the news is based.

About a third of news story headlines and a quarter of news release headlines either misstated the results or went beyond what the research could support. It’s an admittedly small sample of our recent work, but I don’t think the results are out of line with what we see month in and month out. I hope the findings shine a light on the ways that headline messages mislead – and how headline writers can do better.

News story report card: 5 of 16 (31%) headlines overstate study evidence

Credit: Korean Academy of Medical Sciences

Headline: Bloodsucking parasitic hookworms could help make millions of people healthier

Study: Experiments in mice — not people — showed benefits on asthma symptoms.

Our review: “…the story provides no sense of how challenging it will be to translate these results to humans, a very important caveat that should have been discussed.”

Headline: Testing cholesterol in toddlers, even younger? Study says it could help

Study: The study found that a genetic high cholesterol condition was more common in babies than previously thought. It didn’t address whether identifying the condition at a younger age ‘could help’ — an entirely different issue.

Our review: “…the story’s framing takes the stance that finding the risk factors for this disease very early on in life leads to an overall benefit. But the study provides no evidence of that, and parents need to know this.”

Headline: Study offers potential breakthrough in care of children with autism

Study: The study found no statistically significant difference between treatment and control groups for the main outcome — a fact never mentioned in the story.

Our review: “…saying that researchers have found a ‘potential breakthrough’ for results that might have been due to chance seems misleading..”

Headline: How Omega 3 Fats May Improve Fertility

Study: Mice — not people — bred to have higher ratios of omega 3 fatty acids seemed to produce more “precursors to egg cells” and also produced higher quality eggs, meaning they would be more likely to have eggs that would be fertilized and develop into baby mice.

Our review: “The story provides no sense of what’s involved, or just how challenging it will be, to translate these results to humans. So why report it now when there’s no evidence? We don’t know.”

Headline: ‘New day’ in lung cancer as Merck drug shines, works with chemo

Study: The drug delayed progression of often-fatal lung cancers for an additional 4 months compared with chemo in patients whose tumors expressed a certain biomarker. The duration of benefit is never mentioned in the story, nor is the number of lung cancer patients who might be candidates for treatment.

Our review:  “Its ‘new day’ headline and overall optimistic tone may mislead some people into thinking the drug offers lengthy survival (or even a cure) for anyone with this type of often-lethal cancer.”

News release report card: 4 of 17 (24%) headlines overstate evidence

Headline: Easing Labor Pain May Help Reduce Postpartum Depression In Some Women, Early Research Suggests

Study: Epidural use was associated with reduced risk of postpartum depression in this observational study that cannot show cause and effect. Suggesting that pain relief “may help reduce postpartum depression” is not justified by the evidence.

Our review: “The tone of the news release and the abstract (both by the American Society of Anesthesiologists) suggests that epidural anesthesia is helpful in preventing postpartum depression, but that assertion remains far from proven.”

Headline: Cleveland Clinic study finds whole grain diet reduces cardiovascular disease risk 

Study: The study documented a reduction in blood pressure when participants ate more whole grains for 8 weeks. It did not document that this small, short-term reduction in blood pressure had any effect on cardiovascular disease risk or outcomes — a fact never clarified in the release.

Our review: “…the lowered blood pressure was a surrogate outcome and not a primary endpoint, i.e. reduced risk of cardiovascular disease. It would take a much larger outcomes study to demonstrate that.”

Headline: Phase 3 analyses in Alzheimer’s show clinical benefit of tramiprosate in APOE4 carriers

Study: The published study istates that the research — based on a 2,000 patient cohort — did “not reach its primary objectives.” This is never acknowledged in the release. The headline is based on a smaller subgroup analysis.

Our review:  “…this sub-group analysis cannot lead one to draw any conclusions about whether the drug “worked,” given that the overall study was negative.”

Headline: ‘Game-changing’ immunotherapy doubles head and neck cancer survival

Study: Median overall survival was 7.5 months in the treatment group and 5.1 months in the control group. The “game-changing” result amounts to a 2-month increase in survival time.

Our review: “That sort of vague, and potentially misleading, language has little place in a news release like this one.” .

Note: the inspiration for this analysis came from Dr. David Allison’s “Obesity and Energetics Offerings,” a weekly newsletter sent from the University of Alabama, Birmingham.