HealthNewsReview.org | Independent Expert Reviews of News Stories | Holding Health and Medical Journalism Accountable |
|
|
Study: Lots of red meat increases mortality risk
March 24, 2009
One big problem: "lots of red meat increases mortality risk" - suggests that cause-and-effect has been established when this kind of observational study simply CAN NOT establish causation and risk.
Our Review Summary
This story covers a big study that reports a statistical association between a diet high in red meat and a higher death rate.
The report does a number of things well:
- It indicates early on that the results support previous findings and current recommendation about red meat, while adding new data.
- It quotes a variety of sources, including two self-interested representives of the meat industry who question the findings. This attention to the opposition enhances rather than undercuts the report's credibility.
- It translates the findings into dinner-table terms: high consumption in this study is defined as about a quarter pound hamburger per day, low is 5 ounces per week. That's really useful to readers.
The report has two significant shortcomings.
- The language of the story - e.g., "lots of red meat increases mortality risk" - suggests that cause-and-effect has been established when this kind of observational study simply CAN NOT establish causation. It can only point to statistical association. The story never made that clear. So even discussing "risk" is an overstatement.
- Then, when choosing to discuss "risk," it used only relative data ["a 22 percent lower risk of death" by cancer for low-red eaters, for example], not the absolute data [1.3 percent compared to .06 percent]. But once the story started using "risk" terminology without explaining that cause-and-effect and risk cannot be firmly established by such a study, it had already missed the mark.
We know we hold the bar high with this expectation that stories explain the quality of the evidence. But it is not simply splitting hairs; it is inaccurate to use language that does not fit the evidence. Journalists and should consumers should review the excellent guest column by Mark Zweig and Emily DeVoto, entitled "Does Your Language Fit the Evidence?"
One of our medical editors wrote to me about this topic:
"I can tell you, the average practicing physician (one of the presumed target audiences for the medical literature) has a less than firm grasp of this difference! I have been teaching evidence-based medicine for years, and it seems that our residents finally get this by the end, but I don't think it's a skill that was routinely taught in medical training before the past decade or two, nor one that sticks with most docs."None of this denigrates the importance of the study or the quality of the work. But all studies have limitations and news about research should explain those limitations.
Click on Criteria for definitions.
This is a study of subjects' voluntary diets, so no "treatment" is involved.
Again, no treatment is involved. Speculation about the relative costs of high red meat vs. high white meat diets would not be productive.
Because the story confuses association with causation, it indeed does commit disease-mongering repeatedly. It may frighten readers into thinking that cause-and-effect has been established when, in fact, only a statistical association has been established. For example, the story says:
- Headline: "Lots of red meat increases mortality risk." Risk cannot be firmly established by such an observational study.
- "22% higher risk of dying of cancer...." is an example of causal language that isn't warranted with this kind of study finding.
It's a common pitfall of such stories. Association does not equal causation. It's not journalists who get it wrong. Scientists and physicians commonly express this incorrectly. But that doesn't make it acceptable. Most stories we saw on this study stated this incorrectly.
The report is based on results of a large, prospective epidemiological dietary study.
The reporter mentions the important caveats: that the study group may be healthier than the general adult population, and the data is based on self-reports of eating habits, which can be unreliable.
We've already commented on the "association does not equal causation" issue, so we won't rule this criterion unsatisfactory for that flaw.
The story focuses plainly on the death risks of the diets studied. However, risks are stated only in relative terms, not in absolute terms. We won't penalize the story for that here, but we do under "quantification of benefits."
The story makes no claims for the novelty of this kind of research or this general finding.
In the second paragraph the report states that these findings "bolster[s] prior evidence" about the risks of diets high in red meat.
The reporter does a fair job describing the study and the findings.
The article includes such details as size of the study population, the duration of the study and the method of data collection.
But, as already discussed, this kind of study can't establish cause-and-effect or risk. Just statistical associations. However, once it started describing risk, it did that incompletely as well.
It compaes the relative risk of death by heart disease and cancer in the highest- vs. lowest-red meat consumption groups for both men and women.
But the story suffers because it describes the findings only in terms of relative risk--men who ate the most red meat had a 27 percent higher risk of dying from heart disease compared to those who ate the least, for instance.
But the story fails to put these percentages into context. In the example above, about 1.2 percent of the men who ate the most red meat died of heart disease during the 10-year study, compared to .06 percent of those who ate the least red meat
Looking at male deaths by all causes, about 4 percent of the red meat group died, compared to about 2 percent of the least-red-meat group.
Overall, about 13 percent of study participants died during the 10-year study.
This is not to suggest the findings are insignificant. But reporters should always include absolute data to help readers understand the size of the risk an individual faces. (But , as already stated, risk language was risky in this story to begin with.)
Given the number of sources cited, the story does not appear to rely on the Archives of Internal Medicine press release or any other news release.
The reporter does an excellent job sourcing the story. Sources include
- the lead author
- the author of a related editorial
- one independent diet expert
- two industry sources that dispute the findings
The reporter gets extra points for including two industry sources. While their thoughts appear very late in the story, including them both makes the story seem--and actually be--more credible.
There are no financial conflicts to report.
In several places, the article appropiately indicates that limiting red meat in the diet may improve health.
The reporter carefully avoids implications that more extreme options, such as a meatless diet, would cut death risk more.
Total Score: 6 of 8 Satisfactory
The U.S. Preventive Services Task Force is considered the gold standard of preventive health recommendations - including on screening tests. It's a good source for journalists and consumers.
About 70% of the stories reviewed from 2006-9 failed to adequately discuss costs, or to explain how big (or small) are the potential benefits and harms of treatments, tests, products and procedures.
We have documented a disturbing trend of news stories taking an advocacy stance, promoting certain screening tests outside the boundaries of scientific evidence.
Stories on new technologies like Cyberknife, DaVinci robotic surgery systems, and proton beam cancer therapy often fail to scrutinize the evidence and/or to discuss the costs involved.
Rather than suggesting that everyone should be screened for everything, news stories could explain: "All screening tests cause harm; some may do good."
The first 38 network TV network morning health news stories reviewed in 2009 earned an average score of 1.2 stars. 13 of the 38 stories got ZERO stars.
Both TIME magazine and BusinessWeek have published terrific stories explaining the importance of the Number Needed to Treat - or NNT.
Knowing relative risk reduction is like knowing you have a 50% off coupon but not knowing whether it's for a Lexus or a lollipop. Absolute risk reduction tells you what the "coupon" is worth. Read more.
The website NoFreeLunch.org posts "a database of health care professionals who have pledged to accept no gifts from industry and to rely on non-promotional sources of information."
To help journalists cover stories responsibly, we post a list of independent experts who state that they do not have financial ties to drug or medical device manufacturers.
We apply the same ten standardized criteria to the review of every story.
We have about 30 story reviewers. Each story is reviewed by 3 different people.
Gary Schwitzer's seven words you shouldn't use in medical news: cure, miracle, breakthrough, promising, dramatic, hope, victim. Read why.
Our reviewers include two former CNN medical reporters and a former editor of the Washington Post health section.