This story covers a big study that reports a statistical association between a diet high in red meat and a higher death rate.
The report does a number of things well:
The report has two significant shortcomings.
We know we hold the bar high with this expectation that stories explain the quality of the evidence. But it is not simply splitting hairs; it is inaccurate to use language that does not fit the evidence. Journalists and should consumers should review the excellent guest column by Mark Zweig and Emily DeVoto, entitled “Does The Language Fit the Evidence? – Association Versus Causation.”
One of our medical editors wrote to me about this topic:
“I can tell you, the average practicing physician (one of the presumed target audiences for the medical literature) has a less than firm grasp of this difference! I have been teaching evidence-based medicine for years, and it seems that our residents finally get this by the end, but I don’t think it’s a skill that was routinely taught in medical training before the past decade or two, nor one that sticks with most docs.”
None of this denigrates the importance of the study or the quality of the work. But all studies have limitations and news about research should explain those limitations.
Again, no treatment is involved. Speculation about the relative costs of high red meat vs. high white meat diets would not be productive.
The reporter does a fair job describing the study and the findings.
The article includes such details as size of the study population, the duration of the study and the method of data collection.
But, as already discussed, this kind of study can’t establish cause-and-effect or risk. Just statistical associations. However, once it started describing risk, it did that incompletely as well.
It compaes the relative risk of death by heart disease and cancer in the highest- vs. lowest-red meat consumption groups for both men and women.
But the story suffers because it describes the findings only in terms of relative risk–men who ate the most red meat had a 27 percent higher risk of dying from heart disease compared to those who ate the least, for instance.
But the story fails to put these percentages into context. In the example above, about 1.2 percent of the men who ate the most red meat died of heart disease during the 10-year study, compared to .06 percent of those who ate the least red meat
Looking at male deaths by all causes, about 4 percent of the red meat group died, compared to about 2 percent of the least-red-meat group.
Overall, about 13 percent of study participants died during the 10-year study.
This is not to suggest the findings are insignificant. But reporters should always include absolute data to help readers understand the size of the risk an individual faces. (But , as already stated, risk language was risky in this story to begin with.)
The story focuses plainly on the death risks of the diets studied. However, risks are stated only in relative terms, not in absolute terms. We won’t penalize the story for that here, but we do under “quantification of benefits.”
The report is based on results of a large, prospective epidemiological dietary study.
The reporter mentions the important caveats: that the study group may be healthier than the general adult population, and the data is based on self-reports of eating habits, which can be unreliable.
We’ve already commented on the “association does not equal causation” issue, so we won’t rule this criterion unsatisfactory for that flaw.
Because the story confuses association with causation, it indeed does commit disease-mongering repeatedly. It may frighten readers into thinking that cause-and-effect has been established when, in fact, only a statistical association has been established. For example, the story says:
It’s a common pitfall of such stories. Association does not equal causation. It’s not journalists who get it wrong. Scientists and physicians commonly express this incorrectly. But that doesn’t make it acceptable. Most stories we saw on this study stated this incorrectly.
The reporter does an excellent job sourcing the story. Sources include
The reporter gets extra points for including two industry sources. While their thoughts appear very late in the story, including them both makes the story seem–and actually be–more credible.
There are no financial conflicts to report.
In several places, the article appropiately indicates that limiting red meat in the diet may improve health.
The reporter carefully avoids implications that more extreme options, such as a meatless diet, would cut death risk more.
This is a study of subjects’ voluntary diets, so no “treatment” is involved.
The story makes no claims for the novelty of this kind of research or this general finding.
In the second paragraph the report states that these findings “bolster[s] prior evidence” about the risks of diets high in red meat.
Given the number of sources cited, the story does not appear to rely on the Archives of Internal Medicine press release or any other news release.
Systematic, Criteria-Driven
Monday at #ADA2021, an infectious disease epidemiologist, a journalist, and a researcher shared perspectives on communication of risk in the era of COVID-19: http://ow.ly/ROfR30rMeE7 @AstleyCM @garyschwitzer @berthahidalgo @ADA_DiabetesPro
Shoutouts to @susan_bewley @drkevinknopf @MichaelBaum11 in this call for more than fawning coverage from press releases for these kinds of stories.
Also, linked history inside goes back 5 years just on Grail apparently trying to become Holy Grail. https://twitter.com/garyschwitzer/status/1408849696416841731
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like