Coffee & sleep stories mostly miss the association ≠ causation emphasis

Gary Schwitzer is founder and publisher of HealthNewsReview.org.  He gets headaches when he reads most news stories about observational studies, but won’t make a cause-and-effect statement about that.  Follow him on twitter as @garyschwitzer.

Just a quickie.  You’ve heard what follows many times before from us.  It is one of the most common flaws in health care journalism.

Studies that show a statistical association between two things do not necessarily prove that one thing causes another to occur.

We saw that principle violated several times in news coverage of several different studies today – regarding coffee and sleep.

Coffee consumption associated with a lower risk of death

We will have a more in-depth systematic criteria-driven review of one of the coffee stories, but here’s a quick overview.  As usual, the problem resided in the headlines of most stories, with some half-hearted attempts in some stories to include the limitations of two observational studies, published in the Annals of Internal Medicine.  The conclusion of one study emphasized association, not causation:

Higher consumption of coffee was associated with lower risk for death in African Americans, Japanese Americans, Latinos, and whites.

Coffee drinking was associated with reduced risk for death from various causes.
The journal’s “patient page” spelled out some of the limitations of the studies for journalists or the public:

What were the limitations of the studies?

 These studies were based on a single report of how often the participants drank coffee, which may have changed over the years they were followed and might not have been accurate. In addition, although the researchers tried to account for this, people who say they drink a lot of coffee may differ from those who do not in other ways that may affect their health.

An editorial accompanying the published results of the studies in the Annals stated that the protective qualities of coffee was “biologically plausible” but “the association of coffee intake with mortality, however, was modest and sensitive to confounding.” In other words, other things may account for the statistical association.  Cause and effect was not established.  So any headline you see like the following is wrong:

CNN: “Drinking more coffee leads to a longer life, two studies say.”  (“Leads to” implies cause and effect.)

ABC News: “Java drinkers, rejoice: Coffee may help you live longer” (Or it may not. This study didn’t prove anything either way.)

The Guardian: “Coffee cuts risk of dying from stroke and heart disease, study suggests.”  (Cutting risk implies that cause-and-effect has been established.  It has not.)

How could it have been stated better?  Read our primer, “Observational studies: Does the language fit the evidence?

Purposeful life, better sleep

Our associate editor, Jill Adams, who does the daily searches for stories eligible for our review, wrote to our team this morning:

Four of our regularly reviewed outlets failed to make the point that association does not equal causation in covering a recent study that found people with a sense of purpose in life sleep better. The lead researcher certainly didn’t help, saying things like: “Helping people cultivate a purpose in life could be an effective drug-free strategy to improve sleep quality, particularly for a population that is facing more insomnia.”

The study found purposeful subjects reported less sleep-associated problems such as apnea and restless legs. Well geez, perhaps sleep problems lead to difficulties maintaining a sense of purpose.  

The stories she found were:

Newsweek: “HOW TO SLEEP WELL: FILL YOUR LIFE WITH PURPOSE, STUDY SAYS

US News & World Report was one of the news organizations using the researcher quote: “”Helping people cultivate a purpose in life could be an effective drug-free strategy to improve sleep quality, particularly for a population that is facing more insomnia,” (the researcher) added. “Purpose in life is something that can be cultivated and enhanced through mindfulness therapies.”

The Guardian failed to explore the inherent limitations of relying on recall and questionnaires, when it reported: 

“More than 800 people aged 60 to 100 took part in the study and answered questions on their sleep quality and motivations in life. To assess their sense of purpose, the participants were asked to rate statements such as: ‘I feel good when I think of what I’ve done in the past and what I hope to do in the future’.”

And TIME’s headline, “Having a sense of purpose may help you sleep at night,” must be countered with the possibly-equally-true statement, “Or not.”

So, today’s news readers, move on.  There’s probably something in political news today that’s worth more of your time. 

You might also like

Comments (3)

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.

Charles Carter

July 12, 2017 at 5:35 am

Indeed, though similarly unproven, why wasn’t it said that people with better sleep experience more sense of purpose?

Reply

    Gary Schwitzer

    July 12, 2017 at 9:44 am

    Charles,

    Thanks for your note, but we did address a correlate of that possibility, with this line in our article: “perhaps sleep problems lead to difficulties maintaining a sense of purpose.”

    Gary Schwitzer
    Publisher

    Reply

Gayle Greene

July 13, 2017 at 3:43 pm

“Well geez, perhaps sleep problems lead to difficulties maintaining a sense of purpose.” Thank you for pointing this out! As the author of INSOMNIAC and from lifelong experience and from interviewing hundreds of insomniacs, I can assure you, it does!

Reply