Joy Victory is Deputy Managing Editor of HealthNewsReview.org. She tweets at @thejoyvictory.
Last week, HealthDay News (and many other news sites) covered a study from The Lancet Psychiatry looking at the association between exercise and depression. People who said they exercised tended to report fewer days of poor mental health compared to those who said they didn’t exercise, according to the study.
That finding was unfortunately misreported as showing that “exercise really can chase away the blues, to a point.” Why was this misleading? The observational study wasn’t capable of showing cause and effect (or, as statisticians might say, that it wasn’t a “causal” relationship).
It’s the classic chicken-or-egg dilemma. We don’t know, for example, if exercise led to better mental health or better mental health led people to exercise. It’s feasible that both are true, too.
Even though the data set had 1.2 million people, observational findings like these–that can only detect patterns–are reversed all the time when a true experimental, randomized controlled trial (RCT) is conducted. Just last week we wrote about this happening when savvy researchers used both an observational study and an RCT to study workplace wellness programs. While observational data showed the workplace wellness programs got employees to exercise more, the RCT found that wasn’t the case.
That’s not to say there’s isn’t some RCT evidence indicating positive effects of exercise on mental health. There is. However, that research isn’t definitive and generally doesn’t speak to the claims generated by this study, which purports to show specific durations and types of exercise that have the greatest beneficial effects. Such claims simply aren’t supported by the evidence on offer.
In a must-read Twitter thread, PhD student Anne Scheel did an excellent job of making this point and picking apart the multiple problems with this Lancet study and the ensuing news coverage:
So there's a new N = 1.2 MILLION study finding a correlation between physical exercise and mental health (h/t @peder_isager).
Everybody freaking out because…. it's causal, right? Surely it's causal.
And "causal" = exercise –> mental health.
Thread +https://t.co/AIF2H1bcgE— Anne Scheel (@annemscheel) August 13, 2018
Deep into her thread, she revealed one telling detail many journalists didn’t: Adam Chekroud, the lead study author on the Lancet paper, has a significant conflict of interest. He’s the founder of the startup Spring Health, which Chekroud said via a tweet thread is “using these findings to develop personalized exercise plans.” So, in other words, he stands to gain financially by selling plans that are underpinned by these results.
Spring Health, as Scheel points out, retweeted news headlines that overstate the study’s findings. The Lancet also retweeted some misleading news stories.
This wasn’t at all surprising to Darren L Dahly, PhD, an epidemiologist at University College Cork, Ireland.
“And there it is. There’s the bullshit everyone. Noise mine a giant observational dataset. Act like you understand causes, while hand-waving about associations. Then move right onto selling personalized exercise and wellness plans,” he tweeted.
Comments (2)
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Gary Schwitzer
August 21, 2018 at 10:17 amIn another story, the Wall Street Journal buried this caveat in the 12th paragraph:
But, earlier, the story discussed the “effect” of exercise and which types provide “greater benefits.” And the headline makes causal claims: “The Exercise That Helps Mental Health Most: Certain fitness routines do more to help avoid depression, stress or other emotional problems.”
You can’t accurately discuss effect and benefits when the research has not established cause-and-effect.
So the story leaves readers with their heads spinning. One reader left a comment online: “This is a meandering mess – is this what they teach in journalism? Its a mess”
Gary Schwitzer
Publisher
HealthNewsReview.org
Suzanne Holt
August 21, 2018 at 3:54 pmHere’s another problem. According to the columnist the lead author claims a 43% improvement in mental health for exercisers. But the research findings states “43% fewer days of poor mental health in the previous month” (ceteris paribus). These are not equivalent statements. In a 30-day month, exercisers averaged 28 days of non-poor mental health vs. 26.6 days of non-poor mental health for non-exercisers. Those 1.4 days of better health per month is only a 5% improvement, not a 43% improvement.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like