Headline vs. study: Cancer breath, common weeds, and coffee forever!

This is our third go-round looking for disconnects between what a headline implies or emphasizes, and what the substance of the referenced study really is. And we are beginning to see some themes that almost invariably raise our eyebrows, such as headlines that make factual claims from what are clearly preliminary results. Or headlines that–usually in a provocative way–extrapolate far beyond what the data really mean. Finally, one we see so often we have a graphic for it: “Just say no to news stories on mice research.”

I too write headlines and I have to admit that sometimes–in striking a balance between trying to be informative and capturing attention–the latter wins out. So this series is not so much motivated by a policing or scolding attitude as it is a reminder to ourselves and others that headlines that misinform, or even mislead, are not benign. We certainly write headlines to bring readers to our content, but we also have an obligation to deliver that content.

Taken in that context, a headline might just as well be considered a promise we intend to keep.

Since we last looked at headlines vs studies in January, we’ve published 17 new news story reviews, and 18 news release reviews. Here’s how they stack up.

• News STORY report card: 3 of 17 (18%) headlines overstate study evidence (we list two examples below) •

Headline: Common weed could help fight deadly superbug, study finds

Study:  A pre-clinical study of mice looking at whether the Brazilian peppertree plant might contain important antimicrobial properties.

Our review: “As we’ve stressed many times, preliminary rodent research rarely (if ever) rises to the occasion of big health news. We generally recommend putting these studies away and waiting until more conclusive human research comes along.”

Headline: How your morning coffee might slow down aging

Study: The study identified certain genes (or groups of genes) associated with inflammation in older adults. Caffeine was noted to have an inhibitory effect on one gene associated with inflammation. However, the sample size involved in the caffeine-related part of the study was small and only addressed the one gene in particular.

Our review:  “… the news story translates this as meaning that drinking coffee can slow down aging. That’s a stretch. The story also uses vague — and potentially overstated — language to infer that inflammation is the causal factor for a host of diseases. While many chronic diseases are associated with inflammation, there’s little conclusive evidence that lowering inflammation levels — or drinking coffee for that matter — will prevent these diseases.”

• News RELEASE report card: 4 of 18 (22%) headlines overstate evidence (we list three examples below) •

Headline: Phase III COMPASS study with Bayer’s Rivaroxabon in Patients with Coronary or Peripheral Artery Disease shows Overwhelming Efficacy and Meets Primary Endpoint Early

Study: Ostensibly, the study is the largest clinical trial to date on the drug Xarelto (rivaroxaban), used to prevent major adverse cardiac events such as heart attacks and strokes in people with coronary artery or peripheral vascular disease. However, Bayer didn’t release the study–just a news release about it.

Our review : “The release describes the study parameters (number enrolled, dosage, time frame, etc.) and discusses in clear detail the drug’s availability. But it left out potential harms from Xarelto and it also doesn’t cover the drug’s costs, talk about any of the specific benefits of the drug, or compare it to other treatments. In short, it’s like a Superbowl ad for the drug without the all the fast-talking caveats at the end.”

Our reviewers felt this was important because “even when drug makers are required to disclose when [a drug] reaches ‘primary endpoints,’ it’s to the benefit of investors, regulators, the medical community and hopeful patients to avoid hyping the results, emphasize study limitations, and highlight the lack of vetting by peers. That context is going missing in a number of announcements from big drug companies.”

For a more general discussion of data-free drug announcements see Managing Editor Kevin Lomangino’s recent post on a “troubling PR trend.”

Headline: Breath test could help detect stomach and esophageal cancers

Study: Could a simple breath test predict esophageal or stomach cancer, therefore allowing earlier diagnosis and introduction of treatment? Results showed that, in most cases, breath analysis could only distinguish between people who were already known to have stomach or esophageal cancer (advanced in most cases) with people who did not have any signs of cancer.

Our review: “The release did note at the end that researchers have yet to study whether this breath analysis can detect cancers among people in the general population. That’s an important point and one that would have been good to emphasize earlier in the release. The release gave only the briefest nod to the existence of numerous other breath tests for cancer that are being studied and didn’t explain how this particular research compares to other efforts.”

Headline: Encouraging clinical results for an antibody drug to prevent or treat HIV

Study: A preliminary clinical trial of a new biological agent called “10-10704” to treat HIV, involving just 33 people (19 people infected with HIV and 14 who were not).

Our review: “The stated purpose of the research was ‘to determine whether the antibody was safe and whether it had antiviral activity in humans’ but the news release didn’t tell us the actual results of the trial and what this might mean in the larger context of HIV treatment.”

Have you come across any headlines lately that drew you in, but when you read the associated study, left you wondering “where’s the beef?”

If so, let us know in the comments section below.