Alan Cassels is a drug policy researcher at the University of Victoria and a regular contributor to the blog. He is also the author, most recently, of The Cochrane Collaboration: Medicine’s Best-Kept Secret. He tweets as @AKEcassels.
Did you know that growth in the per capita consumption of mozzarella cheese in the U.S. is almost exactly correlated to the rise in Americans who earned their doctorate degrees in civil engineering? Or that the annual tally of the number of people who drowned by falling into a swimming-pool correlates with the number of films Nicolas Cage appeared in?
These are both “true.” And, as in all things worth repeating, let us repeat to ourselves once more: “association does not equal causation.”
The trendy U.S. health news story of the week is whether statins might help prevent Alzheimer’s disease. A mega-observational study published in JAMA Neurology, with nearly 400,000 patients, examined the risk of developing Alzheimer’s among patients who had “high exposure” to statins versus those who had “low exposure.”
The news release sent out to journalists made the results sound conclusive: “Men and women who took statins two years or more lowered their risk of Alzheimer’s disease in the period spanning from 2009 to 2013.” This was followed by an out-of-bounds quote from the lead researcher–a quote that appeared in several news stories we read:
“We may not need to wait for a cure to make a difference for patients currently at risk of the disease. Existing drugs, alone or in combination, may affect Alzheimer’s risk,” said lead and corresponding author Julie Zissimopoulos.
This may have been a soundbite that editors found irresistible, with news stories appearing around the globe, including China, the UK, Singapore, Australia, and the U.S. The wildfire spread of this news demonstrated that axiom of Mark Twain’s that “a lie can travel halfway around the world while the truth is putting on its shoes.”
In the U.S., stories appeared in HealthDay (Whether statins cut Alzheimer’s risk may depend on gender, risk), and replicated in the CBS News under the headline: “Could Statins Help Cut Alzheimer’s Risk?.” A CNN headline asks: “Are statins a key to preventing Alzheimer’s disease?” yet it was the UK’s Daily Mail’s headline that probably wins the prize for exaggeration: “Could statins be the miracle cure for Alzheimer’s? Taking the tablets for just 2 years reduces the risk by up to 15%”
A segment on ABC’s Good Morning America included a fairly even-keeled interview with their medical correspondent, but that was sandwiched between misleading statements from news anchors and the headline that ran across the screen, Breakthrough in fight against Alzheimer’s?
Statins linked to cognitive problems
All the stories above included the caveat that this is not a randomized study and observing statins and the development of Alzheimer’s may not be “causally-related.” The HealthDay story noted, for example, that “the findings don’t prove that statins reduce the chances of developing Alzheimer’s. And if they do shrink the risk, the effect seems to be small.”
In the CNN story, the lead researcher explained “it’s important to know that this is not a study that establishes causality, because it’s not an experiment.”
Yet, on the flip side, many stories buried that caution below statements that make a clear cause-and-effect inference. Most also lacked the larger context on statins, such as the very real controversies that surround their use (including their extensive range of adverse effects, some of which are serious) and the fact that there has been a lot of past research—including randomized trials– that have not found any conclusive causal links between high cholesterol and the development of Alzheimer’s.
Not only that, FDA drug safety communications specifically pointed out “post-marketing reports of cognitive impairment (e.g., memory loss, forgetfulness, amnesia, memory impairment, confusion) associated with statin use.” Of all the stories we read, only CNN mentioned this risk, albeit briefly.
Statins prevent car crashes, AKA ‘healthy user bias’
There was one issue no one seemed to explore: a research effect known as healthy user bias, said Colin Dormuth, a pharmaco-epidemiologist who works for the Therapeutics Initiative at the University of British Columbia in Vancouver.
“The JAMA authors could have studied the occurrence of burns, falls, car accidents, BMD [bone mineral density] tests or eyes exams, and gotten the same results,” Dormuth said.
He referred me to a paper he published in 2009 in Circulation which found that more “adherent” patients (i.e.: those who listen to their doctors and take statins) are less likely to be in car accidents, than non-adherent patients, but it would be absurd to make any claim that statins prevent car crashes.
“You never want to compare statin users to non-statin users in observational studies of protective effects,” he told me, pointedly. This is because of the healthy user effect, in which “statin users are naturally healthier in ways not captured by the data, and so you get spurious protective effects.”
In other words, you would find the healthier people being more careful (less prone to accidents), more likely to get vaccinations and cancer screening, all of which are behaviors more common among healthier people. Incidentally, the “healthy user effect“ is seen in all kinds of observational studies.
Dormuth said he wouldn’t put any faith in any “association” paper related to statins, and points to the conclusion of his paper: “Patients who adhere to statins are systematically more health seeking than comparable patients who do not remain adherent. Caution is warranted when interpreting analyses that attribute surprising protective effects to preventive medications.”
What the numbers really showed
Another area that would have been a rich vein for journalistic treatment would have been looking closer at what the alleged benefits really were, and reporting them in absolute terms.
Most stories defined the benefits this way, saying that the study found “high exposure, defined as taking statins for at least six months in a given year during the study period was associated with a 15 percent decreased risk of Alzheimer’s disease for women and a 12 percent reduced risk for men.”
We know that these 12-15% risk reductions figures are relative numbers, and are of little use. It would have been a lot more informative to have a simple explanation comparing high users, low users and non users of statins, as is found in this bar chart. The numbers, though not adjusted for some risk factors that might influence the magnitude of the associations, provide a general baseline of the real size of the benefit supposedly conferred by statins.
In it, we can see that 1.99% of “non statin users” developed Alzheimer’s disease over the course of the five-year study. Of the statin “low exposure” group, about 1.6% developed the disease versus 1.5% of the high exposure to statins.
When calculated in absolute terms, the risk for developing Alzheimer’s disease is 0.49% lower in high-exposure users (number needed to treat [NNT]=204) and 0.39% for the low-exposure users (NNT=254) compared with non-users. More adherent statin users had about a 0.1% lower risk for Alzheimer’s compared with their less adherent counterparts.
Is this more accurate depiction of the numbers worth world-circulating headlines? These absolute differences of less than half a percent were very small–and hence one could conclude even if there was a substantive causal link, the statins might help about one in 200 people.
‘I would not use the results of this study to guide choices of statins’
Then we get to the splitting of hairs, i.e.: Which race or gender benefits more from taking statins? And which statin works best for this effect?
One of the better conclusions was expressed by Dr. Eric Larson, executive director of the Group Health Research Institute in Seattle. He told CBS/HealthDay: “I would not use the results of this study to guide choices of statins. Things like tolerability, cost and so forth may be more important for many people.”
That pretty well captures it: Taking statins is controversial, and associating their use with other health effects in an observational study is highly misleading. Journalists and readers, take note: The headlines whipping themselves around the world may be bad for everyone’s health.