This WebMD story repeatedly uses language that suggests an active, causal link between chocolate and stroke (e.g. reduce, protect, benefit), when the study — as with all observational studies — was not capable of determining whether chocolate in fact prevents strokes. We provide a primer for journalists and others about how to describe observational studies; reading it could help prevent these problems. Other shortcomings include the use of relative rather than absolute risk comparisons, and lack of a clear description of potential harms. These problems overshadow the story’s bright spots, including the perspective of an independent expert and caveats about the differences between U.S. and Swedish chocolate.
Overstating the results of observational studies is a big problem in health journalism, and it has serious consequences. Since the conclusions of observational studies are frequently overturned by more rigorous research, hyping the results of these early studies can contribute to the sense that researchers are always “flip-flopping” on their recommendations and that they essentially don’t know what they’re talking about. In reality, researchers are usually pretty careful to explain what their studies can and can’t tell us; journalists have to get better at communicating those caveats to readers.
The story does a good job of emphasizing that health benefits are more likely with chocolate that has a higher concentration of cocoa solids. However, these premium chocolates come with a corresponding increase in price compared with your average Hershey Bar– something the story could have explained. Nonetheless, we’ll rule this not applicable as most people probably have a general idea about the cost of chocolate.
The story says that researchers observed a 20% reduction in strokes among participants eating the most chocolate relative to those eating the least — a statistic that conveys very little useful information. Compare this with the much more informative Reuters description that provides absolute risks from data contained in one of the study tables: “Among those with the highest weekly chocolate intake, more than 45 grams, there were 2.5 strokes per 1,000 women per year. That figure was 7.8 per 1,000 among women who at the least, less than 8.9 grams a week.”
Although the story warns that milk chocolate may be “high in sugar, fat, and calories,” it doesn’t spell out the implications of eating too much milk chocolate in an attempt to reap health benefits. (To the contrary, the story suggests that eating larger amounts of U.S. milk chocolate with low cocoa content might yield the same result as eating less of the higher-cocoa Swedish chocolate.) Eating more than a modest amount of chocolate is likely to cause weight gain, which would probably negate any beneficial effects on stroke and increase the risk for other health problems.
Starting with the headline, which claims that “Chocolate May Cut Women’s Stroke Risk,” the story consistently misstates this observational study’s conclusions regarding the relationship between chocolate consumption and stroke. Instead of reporting the study’s finding of an association between chocolate and stroke risk, the story repeatedly suggests that chocolate was responsible for the lower rate of strokes seen in those who ate more chocolate. For example: “The group eating the most chocolate got the most benefit, reducing stroke intake [sic] by 20%.” For comparison, see how the competing Reuters story handled this key limitation by soliciting a quote from one of the study authors, who said: “Given the observational design of the study, findings of this study cannot prove that it’s chocolate that lowers the risk of stroke.”
The story did do a nice job of pointing out that milk chocolate in Sweden, where the study was conducted, is richer in cocoa solids than milk chocolate in the U.S., and so a study conducted here might not report the same findings on stroke risk.
The story didn’t exaggerate the effects of strokes.
The study includes comments from an expert source not affiliated with the study, who provides valuable context on the increased desirability of dark chocolate over milk chocolate.
The story could have included a line about other, more established methods of preventing cardiovascular disease, but didn’t.
The availability of chocolate bars is not in question. But it is important to note that the chocolate sold in Sweden typically has a higher concentration of cocoa solids — the likely source of any protective effect — than chocolate sold in the U.S. The story makes this clear.
The story mentions other research suggesting that chocolate consumption is linked to reduced risk of strokes.
Because the study includes comments from an independent expert, we can be sure it didn’t rely excessively on any press release.
Systematic, Criteria-Driven
The tale of two “negative” studies. Or, how spin is applied to make research findings look more impressive. A must read. https://johnmandrola.substack.com/p/the-tale-of-two-negative-studies
The @nytimes chose to interview one of the editors of the @AnnalsofIM (publisher of this paper) who "gushed with extraordinary glee: It’s huge! There are very few things that reduce your mortality by 30%."
Turns out coffee is not one of those few things despite all that glee. https://twitter.com/garyschwitzer/status/1533202075928215558
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like