I saw this coming as soon as I saw the BMJ news release about a study published in one of its journals, Heart. The BMJ, which seemed to have turned a corner recently, starting to include at least boilerplate news release language about the limitations of observational studies, dropped the ball on a new one.
Larry Husten of Cardiobrief beat me to the punch with his piece, “No, Drinking Coffee Won’t Save Your Life or Prevent Heart Attacks.” Excerpts:
Once again the media has swallowed the bait hook, line, and sinker. Following the publication of a a new study in the journal Heart last night, hundreds of news reports have now appeared extolling the miraculous benefits of coffee. Here’s just one typical headline from the Los Angeles Times: “Another reason to drink coffee: It’s good for your heart, study says.”
But a careful look at the study and previous research on coffee makes clear that this type of reporting is completely unwarranted. As I’ve written previously, the media loves to jump on studies like this and inform the public that, say, intense running is as bad as being sedentary. These sort of upbeat, highly positive stories making simple recommendations based on observational studies that are in no way capable of proving cause and effect are dangerous. Unfortunately, the journalists often receive support from study authors and the journal editors and PR people who encourage this gross misinterpretation and don’t take steps to refute these dangerous misconceptions.
So what went wrong? I think the blame in this case starts with the PR machinery of the journal Heart, which is part of the BMJ (British Medical Journal) group. Here is the headline and subhead (of the BMJ news release) for this paper:
Moderate coffee consumption lessens risk of clogged arteries and heart attacks
People consuming three to five cups of coffee a day have lowest risk of clogging
It seems clear that this sort of headline is an invitation to distortion and exaggeration. I would propose that when publishing epidemiology studies journals and editors be extremely cautious and pro-active in press releases. Undoubtedly the media will continue to make mistakes, but there’s no reason why the medical establishment should be enabling these mistakes.
Examples of headlines to back up Husten’s point:
Coincidentally, another big observational study was published in another journal at the same time. JAMA Internal Medicine published the paper, “Prospective Evaluation of the Association of Nut/Peanut Consumption With Total and Cause-Specific Mortality.” The journal’s news release – while it could have done a better job of emphasizing the limitations of observational studies – nonetheless never crossed a line with its language, sticking with “associated with” or “association” throughout. And the release quoted the authors, who wrote:
“We cannot, however, make etiologic inferences from these observational data.” (My note: Etiologic means assigning a cause.)
And the journal published an accompanying editor’s note by Dr. Mitchell Katz, who wrote:
“Multiple studies have demonstrated the beneficial effects of eating nuts. Nonetheless, the editors felt it was worth publishing another such study for 2 reasons. First, this study combined 3 cohorts to produce a large and diverse sample, including a predominately low socioeconomic cohort of Americans and 2 Chinese cohorts. The authors found that higher nut intake was associated with lower mortality in all 3 cohorts. The consistency of the results between the cohorts and with prior studies that have been performed in higher-income populations increases our confidence that the beneficial effects of nuts are not due to other characteristics of nut eaters.”
I think such an editor’s note provides important perspective. And, in a few words, it captures what I often try to include in my writing when I criticize studies, news releases and news stories for failing to mention the limitations of observational studies. Namely:
Did the editorial and the news release make a difference in news coverage? Maybe. See these examples:
As always, though, many others missed the message, with headlines such as:
News release writers, journalists, and news consumers can learn much more from our primer, “Does the Language Fit the Evidence? – Association Versus Causation.”
Follow us on Twitter: