Health News Review

A study published in Nature Medicine is sending journalists tumbling over each other with enthusiasm for claims that a blood test could help predict Alzheimer’s disease.

Dozens and dozens of stories reported the study with no independent scientific perspective and with little or no discussion of the ethics questions involved in an Alzheimer’s test – when there is currently no effective treatment for the disease.

CNN reported:

“In a first-of-its-kind study, researchers have developed a blood test for Alzheimer’s disease that predicts with astonishing accuracy whether a healthy person will develop the disease.”

Astonishing accuracy?  We’ll come back to that.

CNN also let the lead researcher call it “a potential game-changer.”  (We’ve previously called for a moratorium on that term.) And let him say, “My level of enthusiasm is very high.”  Clearly.  So was CNN’s.

CNN also backed into the huge ethical issues surrounding an Alzheimer’s test – if, indeed, there was one that worked.

“people could get a heads up that they were probably destined to get Alzheimer’s. Although some people might not want to know that they’re destined for a horrible disease, others might be grateful for the warning.

Federoff said he would want to know whether he was on his way to getting the disease, even though there’s nothing he could do about it. He might want to take a family trip he’d been thinking about or might want to appoint a successor at work.

“I would make sure that things that are important to me get done,” he said.

But, Federoff added, others might not want to know they were about to get a devastating disease they were powerless to stop.

“I think it’s a very personal decision,” Federoff said. “It would have to be thought through on multiple dimensions. Patients and their families would have to be counseled.”

But we didn’t hear from any of those people who might not want to know.  We only heard from the lead researcher about what he would choose, and he clearly has a vested interest in promoting this work.

On Fox News, “Medical A Team” member Dr. Marc Seigel called it an “extremely exciting, incredible study….So in the future, people are going to be able to walk into the doctor’s office and get a blood test in advance of this disease and know they’re at risk for it.”  As with the CNN story, the physician-bias for testing came through again as Seigel said, “There’s people out there that say, ‘I don’t want to know…I don’t want to know.’ As a physician, I always say ‘knowledge is power.’ I want to know so I can figure out what to do.”  I wouldn’t put Seigel on my “A team” for this kind of reporting, if we can call it that.

Conversely, NPR put ethics right in the headline, “Alzheimer’s Blood Test Raises Ethical Questions.”

And a far more fundamental question of evidence was raised right in the headline of MedPageToday’s story, “Blood Test for Alzheimer’s Disease — Oversold?“  Immediately in the first paragraph, John Gever wrote that “the accuracy fell short of what would normally be acceptable for a screening test.”  He went on to explain:

“We consider our results a major step toward the commercialization of a preclinical disease biomarker test that could be useful for large-scale screening to identify at-risk individuals,” Federoff said in a Georgetown press release.

But the data did not appear to fully support that optimism. If the study cohort’s 5% rate of conversion from normal cognition to mild impairment or Alzheimer’s disease is representative of a real-world screening population, then the test would have a positive predictive value of just 35%. That is, nearly two-thirds of positive screening results would be false.

In general, a positive predictive value of 90% is considered the minimum for any kind of screening test in normal-risk individuals.”

Contrast that with CNN’s proclamation of “astonishing accuracy.”  The MPT story also posted a sidebar summary called “Action Points” that stated:

“Be aware that the positive predictive value of a test is highly dependent upon the prevalence of disease in the population studied; large-scale screening of healthy adults would lead to an unacceptable false-positive rate, but targeted screening of high-risk individuals may be more efficacious.”

And Gever delivered an independent scientific perspective, which many other stories failed to do:

“Keith L. Black, MD, a neurosurgeon and researcher at Cedars-Sinai Medical Center in Los Angeles, told MedPage Today that it would be important to learn more about how these markers connect with neurological dysfunction.

For the biomarkers found in the study, “it’s difficult to understand how they can be specifically linked to Alzheimer’s disease versus some sort of diffuse neuronal abnormality or connectivity abnormality or synaptic connectivity abnormality,” he said.

“The question, I think, becomes for future studies how well do we understand the underlying biology of it, whether it’s specific to Alzheimer’s disease or whether you might see these same types of changes in these blood biomarkers with other types of diffuse neuronal loss or injury, other types of dementia, or ischemic changes, for example.”

The Behind the Headlines project of the NHS Choices site in the UK also analyzed the study.

(Addendum on March 11: DC’s Improbable Science blog also discussed the limitations of this test and others.)

To show what short or failing memories that journalists themselves have, look at some past Alzheimer’s test breakthrough stories:

——————-

Follow us on Twitter:

https://twitter.com/garyschwitzer

https://twitter.com/healthnewsrevu

and on Facebook.

 

Comments

J Gever posted on March 10, 2014 at 1:36 pm

Thanks for the shout-out. On the ethics issue, I would point out there could be significant value in such a test if it was genuinely accurate, even in the absence of effective treatment. For example, family members could be especially alert for problems related to cognitive impairment such as difficulties with pill-taking, falls, poor financial decisions, etc., and therefore be ready to step in to help with such things.

gad2357 posted on March 10, 2014 at 2:05 pm

Mark Liberman’s of Language Log post “(Mis-) Interpreting medical tests” about the Alzheimer’s study:

http://languagelog.ldc.upenn.edu/nll/?p=11015

Carolyn Kay posted on March 12, 2014 at 5:35 am

What are the ethics of declaring that “there is currently no effective treatment for the disease”, when there certainly are practical measures we can take to prevent, delay, or minimize the effects of cognitive decline (http://bit.ly/18BSLUK).

Surely what you meant to say is “there is currently no pharmaceutical, profit-making treatment for the disease.”

Dan Keller posted on March 12, 2014 at 9:20 pm

John Gever’s coverage of this study points up an important aspect of reporting on screening tests. Just as one should ask for absolute values and not just relative values when comparing data, reporters should ask for the sensitivity (true positive rate) and specificity (true negative rate) of screening tests. These numbers do not have to be included in a story for the lay public, but the reporter should know and understand them to be able to evaluate the performance and utility of the test. As John pointed out, “major steps” involving “astonishing accuracy” may just be slight shuffles involving astonishingly preliminary results.

Laurence Alter posted on March 13, 2014 at 10:25 am

Dear Gary (no staff!):

Television reporting loses again.

Why bother to report on such a losing medium?

Respectfully submitted,

Laurence Alter