Headline vs. study: Predicting, preventing and other clickbait

Predicting and preventing.

These are powerful words that hold strong appeal for most of us, especially when it comes to our health.

But the reality is this: There are very few diseases that can be predicted and prevented. Try to name just one and you’ll see that it’s rarely, if ever, possible.

And yet we continue to get headlines like some of the examples featured below — pitching prediction and prevention, often promising hope or assuaging fear, but usually based on inadequate supporting evidence.

That’s why we regularly take a look back at the news stories and releases we’ve systematically reviewed to see if the headlines and the content are in sync. This is what we found when we looked back over the past month.

News RELEASE report card:  8 of 12 (66%) headlines overstate evidence (we list 3 examples below)

Headline: Extract from soursop leaves can prevent the symptoms of fibromyalgia

Study: Can extract from the tropical plant soursop (aka graviola or guanabana) improve fibromyalgia symptoms in rats?

Our review: “The release feeds rampant misinformation about this plant, falsely announcing that soursop extract is a remedy for chronic pain, anxiety and depression associated with fibromyalgia and can improve the lives of patients with that very common condition. Not until two-thirds of the way into the text do we learn those claims are based on a rat study.”

Headline: First-ever prostate cancer treatment uses gold nanoparticles to destroy tumorous cells

Study: No published study, but an ongoing clinical trial with this research question: Can gold nanoparticles destroy prostate cancer tumors at 3-month follow-up?

Our review: “Light on evidence and heavy on superlatives” with no data provided or any discussion of cost, harms, benefits, or funding. Our reviewers felt that, given the short follow-up, it’s unlikely this study will offer practice-changing evidence.

“Furthermore, based on the enrollment criteria, many of the subjects would have low- to very-low risk prostate cancers — which are best managed with active surveillance. Consequently, the treatment could hardly be described as life-changing.”

Headline: Can’t exercise? A hot bath may help improve inflammation, metabolism, study suggests

Study: Does hot water immersion —  that is, ten 1-hour soaks at 102ºF over two weeks  — affect metabolic and inflammatory biomarkers in the blood of 10 overweight and sedentary men? The answer: Some mild, transient changes were noted that could theoretically be beneficial.

Our review: Yes, exercise can temporarily trigger an anti-inflammatory response at the molecular level, as well as decrease sugar/insulin levels. This study suggests repeated, lengthy hot water immersion can do the same thing in just 10 men studied over 14 days. But that hardly means the two interventions are comparable in any way, or that these blood changes have any meaningful or sustainable clinical benefits.

News STORY report card: 5 of 11 (45%) headlines overstate study evidence (we list 3 examples below)

Headline: Artificial intelligence can predict Alzheimer’s 6 years earlier than medics, study finds

Study: Using an artificial intelligence algorithm to recognize patterns of brain scans in 40 patients, researchers reported being able to predict Alzheimer’s with “100% accuracy” (specificity of 82% was not reported).

Our review: Much larger studies would be needed to know if these results can actually be replicated.

Also, the small cohort was very select and “had already been referred to a memory clinic and their attending neurologist had been concerned enough to order a brain scan. That means it’s completely unknown how well this AI model would predict Alzheimer’s disease in the general public.”

Headline: Five-minute neck scan can spot dementia 10 years earlier, say scientists

Study: An unpublished, observational study presented at a conference suggests increased intensity of “forward compression waves” (FCWs) in the carotid artery is associated with an increased risk of dementia later in life.

Our review: Two key points are excluded: First, this is an observational study that cannot conclude the preliminary findings allow doctors to “spot dementia 10 years earlier.” (Read more about the limitations of observational studies HERE.)

Second, “there is no discussion of the risks of this kind of screening test, notably the risk of false positives and false negatives.”

Headline: There’s a better flu shot

Study: A company-sponsored study of medical records from last year’s flu season that we couldn’t find online, and doesn’t appear to have been published in a peer-reviewed journal.

Our review: An unpublished, company-sponsored observational study based on a single flu season, that fails to provide benefits with absolute numbers, is in no position to promise a “real advance” or a “better flu shot.”

You can find more from our Headline vs. Study series HERE