NOTE TO READERS: When this project lost substantial funding at the end of 2018, I lost the ability to continue publishing criteria-driven news story reviews and PR news release reviews - once the bread-and-butter of the site going back to 2006. The 3,200 archived reviews, while still educational, are getting old and difficult for me to technically maintain on the back end of the website. So I am announcing that I plan to remove these reviews from the site by April 1, 2021. The blog and the toolkit - two of the most popular features on the site - will remain. If you wish to peruse the reviews before they disappear, please do so by the end of March 2021. After that date you may still be able to access them via the Internet Archive Wayback Machine - https://archive.org/web/.
Read Original Story

Diagnosing Alzheimer’s

Rating

2 Star

Diagnosing Alzheimer’s

Our Review Summary

Americans and their news media are in love with tests that tell us about medical problems we already have–as well as ones we might one day develop. These tests are often assumed to be beneficial. This broadcast–about a blood test that may predict the development of Alzheimer’s disease–immediately challenges this assumption. Given that there is no cure for the disease, it asks, “Would you really want to know you have it?”   But by reporting only the “accuracy” of the test, the story overlooks a potentially devastating harm of testing. What happens if the test says you’ll develop Alzheimer’s—and it’s wrong? The researchers report that 6 out of 50 people who did not develop Alzheimer’s tested positive for the disease. In addition, they say the test wrongly tested positive in 1 out of 11 cases who did not have evidence of Alzheimer’s at autopsy. If these numbers prove to be accurate in future studies, as many as 12 out of every 100 people tested would be wrongly told they will develop Alzheimer’s. In addition to the potential emotional and financial toll of an Alzheimer’s diagnosis, these people might also be vulnerable to experimental treatments for a disease they don’t have. The broadcast does not explore this troubling scenario.   The story falls short in others areas as well. It does not mention that any new diagnostic test should be compared against the gold standard—in this case, the examination of brain tissue at autopsy—and that the predictive value of the current blood test is based on a less reliable comparison. The broadcast neglects to note that four of the researchers have financial ties to the company that is developing the test. It does not mention that the research was published as a letter, and not an original research paper. It overlooks mention of other tests currently in use or being studied. Its discussion of costs is incomplete.   Fortunately, the story also makes it clear that we have plenty of time to sort out these issues: The new blood test is “very preliminary” and may require another 5 years of research before it will be ready for wide use. 

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The broadcast cites a doctor from the Mayo Clinic who claims that the test is “cheap” and “inexpensive”. The story does not cite a dollar amount, nor does it compare the cost of the new test to that of current testing. Even if the cost of an individual test can be predicted, there will doubtless be high additional "downstream" costs associated with satisfying the curiosity of all the people at low risk of Alzheimer’s requesting low-yield blood tests.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story states that the test identified Alzheimer’s in 90% of cases (a measure of the test’s “sensitivity”). But it doesn’t state the “specificity” of the test—which includes a measure of false-positive results. This is very important information when weighing the usefulness and potential harms of any diagnostic test. And though the story notes that larger studies will be needed, it fails to point out how many patients the researchers studied. Finally, the story doesn’t discuss whether the researchers compared the new test against the diagnostic gold standard—analysis of brain tissue at autopsy.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Very early on this segment raises a key question about the harms of testing for Alzheimer’s: Given that there is no cure for the disease, “Would you really want to know you have it?” But the story overlooks a potentially devastating harm of testing—what if the test is wrong?  Such harm is almost always a possibility. The researchers report that 6 out of 50 people who did not have Alzheimer’s tested positive for the disease. If these numbers prove to be accurate in future studies, 12 out of every 100 people tested would be wrongly told they will develop Alzheimer’s. In addition to the potential emotional and financial toll of an Alzheimer’s diagnosis, these people might also be vulnerable to experimental treatments for a disease they don’t have. The broadcast does not explore this troubling scenario.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story explains that the test was 90% accurate in “identifying which patients with mild memory loss would go on to develop [Alzheimer’s] within 2 to 6 years.” It also says that larger studies will be needed to confirm this one, pointing out an important limitation of the study. But the story doesn’t discuss other important limitations of the new research. For example, to judge the accuracy of a new diagnostic screen, researchers must compare it against a “gold standard”. The gold standard in Alzheimer’s requires the examination of brain tissue at autopsy. How can the researchers be sure the people deemed to have Alzheimer’s actually have the disease? If they can’t, then the predictive value of the test is uncertain.  The segment also doesn’t mention that the results come from a case-control study with a design that is inherently susceptible to bias, including the selection of the comparison group, or controls.

Does the story commit disease-mongering?

Satisfactory

The story does not exaggerate the effects or risks of Alzheimer’s, and very early raises a key question in the Alzheimer’s screening debate: “Would you really want to know you have it?”

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The broadcast neglects to mention where the research was published (as a letter in the journal Nature Medicine). It also fails to note that four of the authors have financial ties to the company that is developing the test. Although the segment identifies the institutions of the two sources interviewed, it’s impossible to know whether they might have any potential conflicts of interest as well.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The story does not discuss other diagnostic tests for Alzheimer’s that are currently in use or being studied. These include spinal taps and imaging devices.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The story explains that a new blood test for Alzheimer’s disease is “very preliminary” and may require another 5 years of research before it will be commercially available.

Does the story establish the true novelty of the approach?

Satisfactory

The story makes it clear that this is a novel approach to Alzheimer’s diagnosis, though it doesn’t discuss how this differs from other approaches.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story does not appear to rely solely or largely on a press release.

Total Score: 4 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.