Biostatistician Dr. Donald Berry of MD Anderson Cancer Center wrote to me recently, “My assessment of the landscape of observational studies, including much of epidemiology, ranges from bleak to parched earth.”
That should get your attention about why we – all of us who communicate about research findings – need to do a better job when communicating about observational studies.
That includes medical journals and journalists.
And here is a fresh example where both need improvement.
To begin with, let me acknowledge that the BMJ has a huge job, publishing more than 50 specialist journals and having someone write news releases about stuff in those journals. But that’s their choice.
Today, for the second day in a row (here is yesterday’s), I’m going to criticize a BMJ news release on an observational study – something I’d done for years until late last year when it appeared that improvement had arrived. Now it’s clear that it hasn’t arrived.
The Annals of the Rheumatic Diseases is one of those 50+ specialist journals published by BMJ. This week BMJ sent out a news release based on a paper published in that journal. The news release was headlined:
Gout may lessen chances of developing Alzheimer’s disease
The sub-head was:
Study finds 24% lower risk of Alzheimer’s disease amongst people with a history of gout
The 2nd sentence of the release was:
Gout appears to have a protective effect for the brain.
There was not one word in the news release about the limits of such observational data, nor about how this study can not prove that “gout may lessen chances”…cannot prove a “24% lower risk”….cannot prove a “protective effect.”
Placing qualifiers such as “may lessen” or “appears to have” is akin to whispering “I’m not sure” after you’ve just screamed “FIRE!!!”
Why does it matter? Because news stories then follow the lead of the news release and get it wrong.
Let’s be frank: do you really think that journalists regularly read the Annals of the Rheumatic Diseases? No, but the journal news releases are dropped in their email lap.
And then they write about what they see in the news releases.
As the New York Times Well blog did, reporting: “An Upside to Gout: It May Offer Alzheimer’s Protection.” Or it may not, something the blog post never explained.
Look at the confusing jumble of information journalists fall into when they try to report on observational data. Excerpts of the NYT blog:
The reason for the connection is unclear. But gout is caused by excessive levels of uric acid in the blood, and previous studies have suggested that uric acid protects against oxidative stress. This may play a role in limiting neuron degeneration.
“This is a dilemma, because uric acid is thought to be bad, associated with heart disease and stroke,” said the senior author, Dr. Hyon K. Choi, a professor of medicine at Harvard. “This is the first piece of data suggesting that uric acid isn’t all bad. Maybe there is some benefit. It has to be confirmed in randomized trials, but that’s the interesting twist in this story.”
Or, of course, the honest counter-argument to that second-last sentence is that “maybe there is no benefit.” What did this story deliver as actionable information? Remember: this blog is called Well….not Science. The authors of the blog state that the blog “sifts through medical research to help readers live well every day.” How did this piece help readers do that? By encouraging them to get gout? Imagine the new ad campaign in competition with “Got Milk?” Get Gout!
It’s not the first time I’ve criticized the New York Times Well blog for its coverage of observational studies. See “The NY Times Well blog isn’t always so well.” And it probably won’t be the last.
And this was neither the first time (I think it’s the 7th time!) and it appears it won’t be the last time that I’ve criticized news releases by BMJ for how they describe (or fail to describe) observational studies.
Two reminders:
Guaranteed: that’s going to keep us busy.
————————-
Follow us on Twitter:
https://twitter.com/garyschwitzer
Opinions on other issues in news, journals, PR, advertising, marketing
The tale of two “negative” studies. Or, how spin is applied to make research findings look more impressive. A must read. https://johnmandrola.substack.com/p/the-tale-of-two-negative-studies
The @nytimes chose to interview one of the editors of the @AnnalsofIM (publisher of this paper) who "gushed with extraordinary glee: It’s huge! There are very few things that reduce your mortality by 30%."
Turns out coffee is not one of those few things despite all that glee. https://twitter.com/garyschwitzer/status/1533202075928215558
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Comments are closed.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like