On December 31 there may not have been much news. Journalists may have grown tired of “Best of 2013″ or “Top 10 whatever” stories. So a paper in the Journal of the American Medical Association that reports that “patients with mild to moderate Alzheimer’s disease had slower functional decline” after taking big doses of Vitamin E was bound to get a lot of attention.
And it did. More than 200 returns on a web search.
And maybe it shouldn’t have – at least not the kind of attention it garnered.
My 93-year old Dad is showing signs of at least mild cognitive impairment, so believe me, I perk up when I hear Alzheimer’s news. So get this straight: I’m not saying this isn’t important research. I’m not saying it wasn’t newsworthy. I’m simply saying that most of the stories I saw failed to evaluate the evidence, exaggerated benefits and failed to give necessary context and caveats – or the balance of same was sorely out of whack.
My local Star Tribune newspaper put the story in prime real estate – page one above the fold.
Was this study worth that kind of placement? I didn’t see another newspaper in the country give it that same kind of treatment.
But it undoubtedly got that kind of Star Tribune ink and prominence because the study was based at the local VA medical center. And I bet you a lifelong supply of vitamin E pills that if the study had come from neighboring Wisconsin, Iowa, Illinois or the Dakotas, it would not have been page one. It may not have been reported in the Star Tribune at all.
Let me emphasize: if you read the entire Star Tribune 900-word piece, you get a reasonable picture of the state of the research. But readers are led to believe that an editorial weight has been given to a story that gets 900 words starting on page one above the fold.
Yet, buried deep in the story is this line:
An accompanying editorial in JAMA called the results “modest.”
But the editorial stated more:
“…the therapeutic effect seen was modest and more relevant to Alzheimer’s disease symptoms and consequences than to reversal of the disease process.”
The second part of that sentence carries an important point that was not captured in stories whose headlines emphasized slowing the disease process (I’ve added the following emphases):
Boston Globe: Study suggests vitamin E can slow Alzheimer’s
UPI: Vitamin E may slow Alzheimer’s brain decline
HealthDay story on Philadelphia Inquirer site: Daily High-Dose Vitamin E Might Delay Alzheimer’s
Reuters: Vitamin E may slow early Alzheimer’s decline
USA Today: Vitamin E may slow Alzheimer’s decline
And on and on and on in dozens and dozens of stories.
The Associated Press story reflected good sourcing, but those sources framed the news in vastly different ways. For example, on the one hand:
“This is truly a breakthrough paper and constitutes what we have been working toward for nearly three decades: the first truly disease-modifying intervention for Alzheimer’s,’’ said Dr. Sam Gandy of Mount Sinai School of Medicine in New York. ‘‘I am very enthusiastic about the results.’’
And on the other hand:
‘‘It’s a subtle effect but it’s probably real,’’ Dr. Ron Petersen, the Mayo Clinic’s Alzheimer’s research chief, said of the benefit on daily living from vitamin E. ‘‘That has to be weighed against the potential risks’’ seen in earlier studies, he said.
What are readers to make of this?
Breakthrough…first truly disease-modifying intervention?
Or subtle effect that’s “probably” real?
To further confuse/clarify the issue, please note that the researchers reported a 3 point difference in vitamin E users on a 78-point scale of activities of daily living. They write:
“Although there is not a consensus on a minimally clinically important difference for (that assessment), some clinicians, patients, or caregivers would consider a difference of 2 points as meaningful because it potentially represents, for example, a loss of dressing or bathing independently.”
But when you read the study, you see that they reported a “confidence interval” in the score difference of anywhere from .92 (or less than 1 point) on up to 5.39. What that means is that the researchers are confident that the true answer lies somewhere in between. It could be a score change of less than 1 point – which would not be considered meaningful by most observers – or it could mean a 5 point difference. The wider the confidence interval – and this one seems fairly wide to me – the less certain is the estimate of benefit.
I didn’t see a single story that discussed the reported “confidence interval.”
Some readers are not stupid.
A reader on the New York Times website wrote:
“The confidence intervals run by the authors — and particularly the adjusted P values — demonstrate that this study found nothing, and absolutely demonstrated no causation. But that itself is news. Describing the trial as possibly having found a result is disingenuous. The Times’ headline on this article is just plain wrong.”
Though that is harsh, the reader’s attention to the confidence interval is noteworthy. Much more simply stated is this reader comment following a CNN blog post:
“I guess it goes without saying, but if Vitamin E may help Alzheimers it also may not help Alzheimers.”
Some have said that the low cost and apparent safety of Vitamin E might lead people to think, “Why not give this a try?”
Well, a closer look at the accompanying editorial is warranted. The editorialists wrote:
“a meta-analysis of 19 randomized trials (found) that vitamin E in doses greater than 400 IU/d was associated with increased all-cause mortality.”
So the safety question of high-dose vitamin E (2,000 IU/d was the dose in the new study) is still unanswered.
The editorial notes that “the mechanism of action of vitamin E in Alzheimer’s disease is uncertain.” Indeed, the Star Tribune story stated:
Asked how vitamin E might be working, (the principal investigator) laughed and said he had no idea. “We describe it of course as an antioxidant, but I’m hard-pressed to give you any reasonably cogent theory about why it might be doing anything in the brains of patients with Alzheimer’s,” he said.
Finally, though, back to the editorial for an important over-riding observation:
“Major AD (Alzheimer’s disease) treatment trials like this one use functional ability… as an outcome with increasing frequency. The use of functional ability measures for this purpose overtly or tacitly uses impairment in functional ability as though it were solely a consequence of AD progression. Such impairment, however, is not specific to AD but occurs frequently among older people as a consequence of many conditions. Some aspects of this trial highlight the nonspecificity of the link between AD and functional decline.”
Let’s recap what we just learned:
Why didn’t stories wrap up these caveats in one neat little package, instead of once again touting the potential benefits of something slowing symptoms, slowing decline, slowing progression? Most stories had one of these elements or another. But I didn’t see a single story that addressed all of the five bullet points I listed above. Given the growing burden of dementia on an aging population, we must do a better job communicating research results, seeking independent perspectives, and independently vetting claims.
Here are some stories that made noticeable efforts to offer analysis:
A CNN blog post quoted an Alzheimer’s Association statement:
“The results are positive enough to warrant more research to replicate and confirm these findings, but should not change current medical practice. No one should take vitamin E for Alzheimer’s except under the supervision of a physician.”
And it had a clearly labeled “Caveats” heading, noting:
Another sound summary came on the aforementioned New York Times “The New Old Age” blog:
But other studies have found that vitamin E failed to delay dementia in people without symptoms or with mild cognitive impairment, which may precede Alzheimer’s.
“It was dead stone cold in the M.C.I. trial,” said the leader of that study, Dr. Ronald Petersen, director of the Mayo Clinic’s Alzheimer’s center. “You couldn’t have found a closer match to placebo.”
Dr. Denis Evans, a professor of internal medicine at Rush University, who wrote an editorial accompanying the new study, cautioned against extrapolating the results to anyone without mild to moderate Alzheimer’s.
“Does this mean that all of us who don’t want to develop Alzheimer’s should rush out and purchase a bottle of vitamin E?” he said. “Oh, please don’t.”
A Wall Street Journal story put this in the second sentence:
“However, the research… found no impact on memory and doctors said there was no evidence that vitamin E prevents the debilitating disease.”
Kudos to those who tried to add independent analysis – even on New Year’s Eve – either by story framing/emphasis, by use of independent experts, and/or by offering context and caveats on their own.
Here’s a list of just a few past examples of less-than-optimal Alzheimer’s disease news coverage:
Follow us on Twitter: