Our Review Summary
This story covers a big study that reports a statistical association between a diet high in red meat and a higher death rate.
The report does a number of things well:
- It indicates early on that the results support previous findings and current recommendation about red meat, while adding new data.
- It quotes a variety of sources, including two self-interested representives of the meat industry who question the findings. This attention to the opposition enhances rather than undercuts the report’s credibility.
- It translates the findings into dinner-table terms: high consumption in this study is defined as about a quarter pound hamburger per day, low is 5 ounces per week. That’s really useful to readers.
The report has two significant shortcomings.
- The language of the story – e.g., “lots of red meat increases mortality risk” – suggests that cause-and-effect has been established when this kind of observational study simply CAN NOT establish causation. It can only point to statistical association. The story never made that clear. So even discussing “risk” is an overstatement.
- Then, when choosing to discuss “risk,” it used only relative data [“a 22 percent lower risk of death” by cancer for low-red eaters, for example], not the absolute data [1.3 percent compared to .06 percent]. But once the story started using “risk” terminology without explaining that cause-and-effect and risk cannot be firmly established by such a study, it had already missed the mark.
We know we hold the bar high with this expectation that stories explain the quality of the evidence. But it is not simply splitting hairs; it is inaccurate to use language that does not fit the evidence. Journalists and should consumers should review the excellent guest column by Mark Zweig and Emily DeVoto, entitled “Does The Language Fit the Evidence? – Association Versus Causation.”
One of our medical editors wrote to me about this topic:
“I can tell you, the average practicing physician (one of the presumed target audiences for the medical literature) has a less than firm grasp of this difference! I have been teaching evidence-based medicine for years, and it seems that our residents finally get this by the end, but I don’t think it’s a skill that was routinely taught in medical training before the past decade or two, nor one that sticks with most docs.”
None of this denigrates the importance of the study or the quality of the work. But all studies have limitations and news about research should explain those limitations.
Criteria
Not Applicable
Again, no treatment is involved. Speculation about the relative costs of high red meat vs. high white meat diets would not be productive.
Not Satisfactory
The reporter does a fair job describing the study and the findings.
The article includes such details as size of the study population, the duration of the study and the method of data collection.
But, as already discussed, this kind of study can’t establish cause-and-effect or risk. Just statistical associations. However, once it started describing risk, it did that incompletely as well.
It compaes the relative risk of death by heart disease and cancer in the highest- vs. lowest-red meat consumption groups for both men and women.
But the story suffers because it describes the findings only in terms of relative risk–men who ate the most red meat had a 27 percent higher risk of dying from heart disease compared to those who ate the least, for instance.
But the story fails to put these percentages into context. In the example above, about 1.2 percent of the men who ate the most red meat died of heart disease during the 10-year study, compared to .06 percent of those who ate the least red meat
Looking at male deaths by all causes, about 4 percent of the red meat group died, compared to about 2 percent of the least-red-meat group.
Overall, about 13 percent of study participants died during the 10-year study.
This is not to suggest the findings are insignificant. But reporters should always include absolute data to help readers understand the size of the risk an individual faces. (But , as already stated, risk language was risky in this story to begin with.)
Satisfactory
The story focuses plainly on the death risks of the diets studied. However, risks are stated only in relative terms, not in absolute terms. We won’t penalize the story for that here, but we do under “quantification of benefits.”
Satisfactory
The report is based on results of a large, prospective epidemiological dietary study.
The reporter mentions the important caveats: that the study group may be healthier than the general adult population, and the data is based on self-reports of eating habits, which can be unreliable.
We’ve already commented on the “association does not equal causation” issue, so we won’t rule this criterion unsatisfactory for that flaw.
Not Satisfactory
Because the story confuses association with causation, it indeed does commit disease-mongering repeatedly. It may frighten readers into thinking that cause-and-effect has been established when, in fact, only a statistical association has been established. For example, the story says:
- Headline: “Lots of red meat increases mortality risk.” Risk cannot be firmly established by such an observational study.
- “22% higher risk of dying of cancer….” is an example of causal language that isn’t warranted with this kind of study finding.
It’s a common pitfall of such stories. Association does not equal causation. It’s not journalists who get it wrong. Scientists and physicians commonly express this incorrectly. But that doesn’t make it acceptable. Most stories we saw on this study stated this incorrectly.
Satisfactory
The reporter does an excellent job sourcing the story. Sources include
- the lead author
- the author of a related editorial
- one independent diet expert
- two industry sources that dispute the findings
The reporter gets extra points for including two industry sources. While their thoughts appear very late in the story, including them both makes the story seem–and actually be–more credible.
There are no financial conflicts to report.
Satisfactory
In several places, the article appropiately indicates that limiting red meat in the diet may improve health.
The reporter carefully avoids implications that more extreme options, such as a meatless diet, would cut death risk more.
Not Applicable
This is a study of subjects’ voluntary diets, so no “treatment” is involved.
Satisfactory
The story makes no claims for the novelty of this kind of research or this general finding.
In the second paragraph the report states that these findings “bolster[s] prior evidence” about the risks of diets high in red meat.
Satisfactory
Given the number of sources cited, the story does not appear to rely on the Archives of Internal Medicine press release or any other news release.
Total Score: 6 of 8 Satisfactory
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like