The story’s suggestion that the spinal fluid analysis used in the study is 100 percent accurate is, in fact, inaccurate. (The "can be 100% accurate" phrase is misleading and unhelpful.) The specificity of the test – about 1/3 of those who tested positive had no evidence of Alzheimer’s – is a big issue.
Changing a few words in the headline and lead sentence of this story (or simply skipping straight to the second sentence) would have given readers a far more balanced portrayal of the results of a study of biomarkers for Alzheimer’s Disease, but as it stands the story is likely to mislead many readers into believing that a “100 percent accurate” test for predicting Alzheimer’s Disease is available.
The study is newsworthy because it indicates that a new tool may be coming to help researchers testing potential treatments for Alzheimer’s. Clinical tests that could be developed based on these study results may also be useful to help reduce some uncertainty for individuals who have been given a diagnosis of probable Alzheimer’s Disease. However, as the researchers point out in their journal article in the Archives of Neurology, many years of study need to be done to find out what sort of predictive power this sort of test may offer healthy people. Readers of this story could be forgiven for mistakenly believing that day has already arrived.
Confirming a diagnosis for a person who is already sick is quite different from predicting the future for a healthy person. Diagnostic testing for any disease, especially one with the emotional and societal impact of Alzheimer’s, is an important public health issue. Because of the importance of the disease, special care should be taken in reporting of potential new development. Spinal fluid analysis as a method for diagnosing Alzheimer’s has been under development for a number of years and while the technology continues to improve, the available testing methods are still lacking in term of their sensitivity (ability to diagnose the disorder when it exists) and specificity (ability to only diagnose those with the disorder. This is especially true for diagnostic tests that are intended to identify those with Alzheimer’s before symptoms are evident.
Not only does this story fail to mention the costs of spinal taps, PET scans or other potential tests for biomarkers of early Alzheimer’s Disease, it does not alert readers to the possible financial consequences of a test indicating that a person is likely to develop Alzheimer’s Disease, such as problems obtaining health, life or disability insurance, job discrimination or other consequences. Although recent federal legislation protects Americans against discrimination based on genetic information, the law does not apply to spinal taps, PET scans or other biomarker tests that do not reveal genetic information.
Perhaps the NYT will pursue this as a next-day folo.
Genetic Information Nondiscrimination Act (GINA) of 2008
The body of this story points out that the spinal fluid test for biomarkers for Alzheimer’s Disease will be very useful to researchers. It also outlines some of the limitations of the test, particularly since there is no known effective treatment. However, the headline that the test can “predict Alzheimer’s” and the lead sentence stating that the test is “100 percent accurate” will likely mislead most readers. Although the story is careful to note that the “accuracy” claim involved only the subset of study participants who already had significant memory loss and who then went on to develop Alzheimer’s Disease, readers are likely to mistakenly believe that the test was “100 percent accurate” at predicting Alzheimer’s Disease. Indeed, while the researchers did report that all of the 57 participants who progressed from mild cognitive impairment to Alzheimer’s Disease during the study did have the biomarker pattern associated with Alzheimer’s, there were another 85 participants with impairment and the “disease” pattern who did not progress during the study.
The body of the story notes many of the limitations of the information this test provides, but those caveats are overshadowed by the strong wording of the headline and lead sentence.
Correction and addendum on August 16:
The “100%” figure comes not from the subjects in this current study, but from a validation test that used data from a different study. We mistakenly thought the 57 subjects who went from impairment to Alzheimer’s were part of the 142 impaired (MCI) subjects in the main study who had the Alzheimer’s biomarker pattern. Actually, the 57 subjects were part of a completely separate study reported in Lancet Neurology in 2006 that was used to validate the biomarker pattern in this study. The researchers looked at 57 subjects who progressed from mild cognitive impairment to Alzheimer’s over a five-year period in that separate study. While they did report that spinal fluid from all 57 of those subjects did contain the ‘disease pattern’ of biomarkers in their spinal fluid, there were actually 137 participants whose spinal fluid was collected in that second study. The researchers did not report the biomarker patterns from 80 participants who did not progress to Alzheimer’s during that study, so we do not know if any of those individuals had the ‘disease pattern’ of biomarkers.
In addition, the researchers did report the results of a second validation test using a group of subjects who had Alzheimer’s confirmed during autopsies. The spinal fluid biomarkers in those subjects showed a 93 or 94 percent match to the ‘disease pattern’ depending on the model used.
Also, it is worth noting that these biomarker patterns are not black-or-white, yes-or-no results. Individual results are placed along a spectrum and there is a gray zone between those that fit the definition of the ‘disease pattern’ in this study and those who fit in the ‘healthy pattern.’ As this sort of testing eventually moves out of small research studies into clinical use involving perhaps millions of people, that gray zone will contains multitudes.
The term "100%" exudes certainty. The reality of this test is far from that ideal.
This rating is a close call. The story does highlight concerns about testing people for a disease for which there is no effective treatment. It also obliquely refers to the risk of erroneous results mislabeling people as destined to develop Alzheimer’s Disease. However, the story includes comments from experts who downplay the risks to spinal taps without noting the number of people who experience severe pain, bleeding, infections or other problems. The discussion of potential harms seems imbalanced.
This story accurately summarizes the most recent study and outlines some of the hurdles that would need to be cleared before a spinal tap test for Alzheimer’s Disease would be ready for regular clinical use. However, by saying the test “can be 100 accurate” and including a quote calling it “the bull’s-eye of predictive accuracy,” the story misleads readers and obscures the fact that many participants in the study who had spinal fluid test results that were like those of Alzheimer’s Disease patients did not actually develop memory problems during the course of the study.
The story should have more clearly explained that just because the test found that almost everyone who developed Alzheimer’s during the study had certain biomarkers in their spinal fluid does not prove that a person with normal memory who has those same biomarkers will develop Alzheimer’s Disease within a certain number of years.
This story does not exaggerate the prevalence or seriousness of Alzheimer’s Disease. However, by incorrectly stating that the test can identify people who are "on their way to developing Alzheimer’s disease," the story encourages confusion between risk factors for a disease and the disease itself.
This story includes comments from independent sources, but it fails to note the industry connections of key researchers and the industry funding of the study.
Several of the researchers who were responsible for the conduct of the study and the interpretation of the results are or were employees of a company (Innogenetics) that produces key components of the spinal fluid test and thus would benefit from clinical use of such a test. Also, funding for the study came from an institute (the Alzheimer’s Disease Neuroimaging Institute) that is a partnership between public and private entities and that has received more than $20 million from pharmaceutical companies involved in developing potential tests and treatments for Alzheimer’s Disease, as well as from the Alzheimer’s Association and some other groups.
The story notes that brain scans are being developed that might also help confirm diagnoses of Alzheimer’s Disease.
The story states that spinal fluid tests for Alzheimer’s are commercially available. However, the test used in the study that was the focus of this story used chemicals that are available only to researchers. Also, as a review of spinal fluid biomarkers for Alzheimer’s disease noted, almost all the testing done to date has taken place in careful controlled research labs and there is substantial work to be done to ensure that different labs could produce consistent results.
Lessons from Multicenter Studies on CSF Biomarkers for Alzheimer’s Disease
The story appropriately portrays this study as a notable advance in the ability of researchers to tell when serious memory impairment is probably caused by Alzheimer’s Disease and not some other form of dementia or brain disorder.
The story does not appear to rely on a news release.