A word about limitations of observational studies and a mention of the vast amount of past research into caffeine and dementia were warranted — but omitted.
This news release, promoting research findings published in the peer-reviewed Journals of Gerontology: Series A, describes what researchers from the University of Wisconsin conclude is a significant positive relationship in older women between reduced odds of getting some form of dementia and consuming the amount of caffeine contained in three or more eight ounce cups of coffee a day. The release highlights the findings as contributing to “mounting evidence” of caffeine’s “potentially protective” effects against dementia. It suggests caffeine consumption is very safe; the fact that the conclusions are based on “self-reported” caffeine consumption levels; and how the researchers adjusted for potentially confounding risk factors.
Several news organizations, including a Milwaukee newspaper, wrote articles based largely (sometimes almost verbatim) on the release, although most did at least make clear that the research in no way proves that drinking coffee or other heavily caffeinated beverages prevents dementia. That’s because the study is observational. As our tip for reporting on observational studies points out: “Because observational studies are not randomized, they cannot control for all of the other inevitable, often unmeasurable, exposures or factors that may actually be causing the results. Thus, any “link” between cause and effect in observational studies is speculative at best.”
The news release left out that caveat, along with known ill effects of moderately heavy caffeine intake, especially among older people. Also missing was acknowledgement of the mixed results of previous non-human animal and association studies involving coffee and cognitive impairment.
“Whiplash” nicely describes the impact of decades of contradictory and sometimes muddled reports about the protective benefits or harms of caffeine consumption. Review articles, such as this one in the Journal of Alzheimer’s Disease, and this story in the Los Angeles Times, have done a nice job of describing the ups and downs of the debate over coffee consumption, in particular. Healthcare organizations often advise no more than a 500 mg/day level of intake to keep away increased risk of insomnia, nervousness, rapid heartbeat, upset stomach and muscle tremor, while the level conferring the “36 percent” risk reduction cited in the news release occurred in women over 65 drinking more than 261 mg a day.
People who already drink coffee or other caffeinated beverages are always eager to have their habits affirmed as “healthy,” while those who don’t are often interested in the “caffeine wars” to inform their opinions about adding colas and coffee to their diets to protect against dementia and a whole host of diseases from diabetes to stroke to liver cancer.
Cognitive decline is increasing in prevalence and society is looking for lifestyle changes that reduce risk. Therefore, reports of new research have baked-in widespread appeal and more than usual obligations to put the work in context.
Caffeine is already ubiquitous and people consume it daily in coffee, tea, sodas and supplements. A discussion of costs isn’t necessary.
The release states that among older women in a large study, “self-reported caffeine consumption of more than 261 mg per day was associated with a 36 percent reduction in the risk of incident dementia over 10 years of follow-up.”
The release would have been better with some explanations of what the 36 percent reduction in risk of incident dementia actually meant.
The published study doesn’t include any numbers in the primary tables to allow for a calculation of absolute risk reduction. The only statistics provided, even in the body of the paper, are the Hazard Ratios, including the 0.76, which gives rise to the 36 percent reduction for the higher caffeine consumers. In this instance, the release writer may have done their best with the available information from the paper, but the writer could have consulted with the study authors to provide this important additional detail. At the very least, providing some estimate of the baseline risk in the study would have helped put the 36 percent reduction in perspective. If that baseline risk was low to begin with, then a 36 percent reduction might not be very meaningful.
Does the news release adequately explain/quantify the harms of the intervention?
Nary a mention of the potential risks of caffeine, particularly in an older population. Caffeine is associated with headaches, tachycardia (rapid heart rate), and anxiety to name a few.
The release earns a Not Satisfactory for not mentioning the study limitations, and in this case the big one is that it was an observational cohort study, a type of study design that does not allow for determination of causation. The issue is that even after statistical adjustment for con-founders (e.g. age, race, education), there are still likely to be factors that occur along with caffeine consumption that are associated with cognitive decline. For example, there may be a genetic predisposition to enjoying the taste of coffee and tea, and being protected from cognitive decline. So in fact the caffeine may have nothing to do with the cognitive decline, it is just a marker for the genetic makeup of the individual.
On the plus side, the release explains that the data come from participants in the well-regarded Women’s Health Initiative Memory Study, funded by the National Institutes of Health; and that the research is composed of information from more than 6,400 women. It also notes that the researchers adjusted for risk factors such as race, age, hormone use, education, obesity, depression, smoking, alcohol use and cardiovascular disease.
No mongering here.
The release notes that the study was funded by the National Heart, Lung, and Blood Institute, National Institutes of Health.
Our expectation is that the release would at least share some basic alternatives. For example, this National Institute on Aging education page states regular aerobic exercise is associated with reduced risk of cognitive decline, and that many other lifestyle and dietary factors (such as a plant-based diet) are believed to reduce risk.
The release quotes the principal investigator saying, essentially, that caffeine is an easy thing to add to the diet and is widely available. True enough.
The release claims that the study is “unique” in that it is based on an “unprecedented” opportunity to study the relationship between caffeine and dementia in a “large and well defined” ongoing study. The hyperbole may be justified, but there is no evidence that it is from the information given in the release.
In fact, there have been a large number of widely reported observational studies and literature reviews examining these associations over the last decades.
Beyond the inflated comments about novelty stated above, the release doesn’t rely on sensational, unjustified language.
This news release, promoting research findings published in the peer-reviewed Journals of Gerontology: Series A, describes what researchers from the University of Wisconsin conclude is a significant positive relationship in older women between reduced odds of getting some form of dementia and consuming the amount of caffeine contained in three or more eight ounce cups of coffee a day. The release highlights the findings as contributing to “mounting evidence” of caffeine’s “potentially protective” effects against dementia. It suggests caffeine consumption is very safe; the fact that the conclusions are based on “self-reported” caffeine consumption levels; and how the researchers adjusted for potentially confounding risk factors.
Several news organizations, including a Milwaukee newspaper, wrote articles based largely (sometimes almost verbatim) on the release, although most did at least make clear that the research in no way proves that drinking coffee or other heavily caffeinated beverages prevents dementia. That’s because the study is observational. As our tip for reporting on observational studies points out: “Because observational studies are not randomized, they cannot control for all of the other inevitable, often unmeasurable, exposures or factors that may actually be causing the results. Thus, any “link” between cause and effect in observational studies is speculative at best.”
The news release left out that caveat, along with known ill effects of moderately heavy caffeine intake, especially among older people. Also missing was acknowledgement of the mixed results of previous non-human animal and association studies involving coffee and cognitive impairment.
“Whiplash” nicely describes the impact of decades of contradictory and sometimes muddled reports about the protective benefits or harms of caffeine consumption. Review articles, such as this one in the Journal of Alzheimer’s Disease, and this story in the Los Angeles Times, have done a nice job of describing the ups and downs of the debate over coffee consumption, in particular. Healthcare organizations often advise no more than a 500 mg/day level of caffeine intake to keep away increased risk of insomnia, nervousness, rapid heartbeat, upset stomach and muscle tremor, while the level conferring the “36 percent” risk reduction cited in the news release occurred in women over 65 drinking more than 261 mg a day.
People who already drink coffee or other caffeinated beverages are always eager to have their habits affirmed as “healthy,” while those who don’t are often interested in the “caffeine wars” to inform their opinions about adding colas and coffee to their diets to protect against dementia and a whole host of diseases from diabetes to stroke to liver cancer.
Cognitive decline is increasing in prevalence and society is looking for lifestyle changes that reduce risk. Therefore, reports of new research have baked-in widespread appeal and more than usual obligations to put the work in context.
Caffeine is already ubiquitous and people consume it daily in coffee, tea, sodas and supplements. A discussion of costs isn’t necessary.
The release states that among older women in a large study, “self-reported caffeine consumption of more than 261 mg per day was associated with a 36 percent reduction in the risk of incident dementia over 10 years of follow-up.”
The release would have been better with some explanations of what the 36 percent reduction in risk of incident dementia actually meant.
The published study doesn’t include any numbers in the primary tables to allow for a calculation of absolute risk reduction. The only statistics provided, even in the body of the paper, are the Hazard Ratios, including the 0.76, which gives rise to the 36 percent reduction for the higher caffeine consumers. In this instance, the release writer may have done their best with the available information from the paper, but the writer could have consulted with the study authors to provide this important additional detail. At the very least, providing some estimate of the baseline risk in the study would have helped put the 36 percent reduction in perspective. If that baseline risk was low to begin with, then a 36 percent reduction might not be very meaningful.
Nary a mention of the potential risks of caffeine, particularly in an older population. Caffeine is associated with headaches, tachycardia (rapid heart rate), and anxiety to name a few.
The release earns a Not Satisfactory for not mentioning the study limitations, and in this case the big one is that it was an observational cohort study, a type of study design that does not allow for determination of causation. The issue is that even after statistical adjustment for con-founders (e.g. age, race, education), there are still likely to be factors that occur along with caffeine consumption that are associated with cognitive decline. For example, there may be a genetic predisposition to enjoying the taste of coffee and tea, and being protected from cognitive decline. So in fact the caffeine may have nothing to do with the cognitive decline, it is just a marker for the genetic makeup of the individual.
On the plus side, the release explains that the data come from participants in the well-regarded Women’s Health Initiative Memory Study, funded by the National Institutes of Health; and that the research is composed of information from more than 6,400 women. It also notes that the researchers adjusted for risk factors such as race, age, hormone use, education, obesity, depression, smoking, alcohol use and cardiovascular disease.
The release notes that the study was funded by the National Heart, Lung, and Blood Institute, National Institutes of Health.
Our expectation is that the release would at least share some basic alternatives. For example, this National Institute on Aging education page states regular aerobic exercise is associated with reduced risk of cognitive decline, and that many other lifestyle and dietary factors (such as a plant-based diet) are believed to reduce risk.
The release quotes the principal investigator saying, essentially, that caffeine is an easy thing to add to the diet and is widely available. True enough.
The release claims that the study is “unique” in that it is based on an “unprecedented” opportunity to study the relationship between caffeine and dementia in a “large and well defined” ongoing study. The hyperbole may be justified, but there is no evidence that it is from the information given in the release.
In fact, there have been a large number of widely reported observational studies and literature reviews examining these associations over the last decades.
Beyond the inflated comments about novelty stated above, the release doesn’t rely on sensational, unjustified language.
Systematic, Criteria-Driven
As always, encouraging news outlets covering medicine to practice journalism, not stenography. https://twitter.com/garyschwitzer/status/1267796903141539846
Thanks @garyschwitzer & @HealthNewsRevu
So helpful & rational as always:
"Often it is what journalists allow interviewees to say – unchallenged – that is most concerning" https://twitter.com/garyschwitzer/status/1267796903141539846
CBS @60Minutes promoted one hospital’s “promise of plasma” for #COVID19. Here are some things that you need to know about that story. https://www.healthnewsreview.org/2020/06/60-minutes-promotes-one-hospitals-promise-of-plasma/
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like