The story focuses on a forthcoming study that aims to evaluate if virtual reality technology can be used to identify people who will eventually develop Alzheimer’s disease.
The story’s headline states: “Virtual reality to help detect early risk of Alzheimer’s.” That is misleading. The researchers haven’t even recruited study participants yet, much less conducted the study or analyzed the results. The story also fails to describe how potential diagnosis might work, what the technique may cost, or what risks are involved. It’s also not clear whether any of the researchers have a potential conflict of interest. In short, the story raises far more questions than it answers. That’s problematic.
Alzheimer’s disease affects around 5 million Americans — and that number is expected to double by 2060. The disease adversely affects the quality of life for both patients and their loved ones, and it is the sixth leading cause of death in the United States. In short, Alzheimer’s disease is a big deal, and any new advances in risk assessment or diagnosis are newsworthy.
However, because this is a subject that many people are passionate about, it is especially important to approach new research responsibly. In this instance, the study has not even taken place yet, and the story fails to answer most of the major questions one might have regarding Alzheimer’s research.
We can understand that a research institution may want to raise awareness of a forthcoming study in order to recruit study participants. We are less clear on why a major news outlet would write about the research — which hasn’t even taken place yet — as if it is already newsworthy and generating data that clinical decisions can be based upon.
Cost is not discussed. And there are multiple costs to consider here: the cost of the virtual reality (VR) hardware itself, the cost of whatever software the study participants would be using, and the cost of training professionals to interpret the data generated by study participants using the virtual reality technology. (Note: we say “study participants” because there are no patients yet, but — ultimately — the cost to patients is what we’d be interested in.)
This is tricky. The study has not yet taken place, so there are no benefits to discuss — which makes it difficult to ding the story for failing to quantify the benefits. As such, we’ll rate this as not applicable. However, our reasons for rating it N/A highlight a critical problem with the story, which we address in Quality of Evidence and Availability, below.
The big risk here, as with any screening test, is incorrect results. Even if the study hasn’t happened yet, and important data sets like sensitivity and specificity can’t yet be quantified, they should be addressed in the story. What are sensitivity and specificity? Sensitivity measures how often a test correctly generates a positive result for people who have the condition that’s being tested for. Specificity measures a test’s ability to correctly generate a negative result for people who don’t have the condition that’s being tested for. In other words, there are two possible harms inherent in diagnostic and risk assessment tools: the failure to identify someone who has Alzheimer’s; and the “false positive” misdiagnosis of people who do not have Alzheimer’s. Either mistake can have significant costs — financial and personal — for patients.
The story offers a rough overview of the size of the forthcoming study. But it fails to address key points about the study. How will the study assess a patient’s navigation skills (which the point of using a VR device)? How will it distinguish between people whose navigation skills are poor because of early neural degeneration, versus people who have never been good at navigating? Is this a longitudinal study, with patients participating in the study over the course of years, coupled with long-term follow-up to determine if and when patients develop Alzheimer’s or other, related conditions? Those are all good questions, and they are not answered here.
The story notes that the forthcoming study has received funding from the Alzheimer’s Society. However, the only source cited in the story is one of the researchers who will be conducting the study. Input from an independent expert about the viability of such a study would have been very welcome.
There is no discussion of other approaches that are used to identify people who have Alzheimer’s disease among high-risk groups. There is also no acknowledgement of the fact that there is no definitive way to diagnose Alzheimer’s disease in patients while they are still alive (the only definitive diagnostic technique involves an autopsy). That’s a heck of an oversight.
On the one hand, the story does tell readers that there will be a forthcoming study to evaluate whether VR technology can be used to test people’s navigation ability in order to determine whether it is an effective technique for diagnosing Alzheimer’s. (In other words, they don’t even know if this is something that will work yet.)
On the other hand, earlier in the story, readers run across sentences like: “Virtual reality to help detect early risk of Alzheimer’s” and “Those who do worse in the tests will be the ones most likely to succumb to Alzheimer’s later in life, scientists now believe.” That makes it sound like this is already a useful tool, and availability could be right around the corner. Ultimately, the story never explicitly tells readers that any possible VR tool is years away, at best. That tips the scales for us, and we lean in the direction of “not satisfactory” on this criterion. One wonders how many people read this story and contacted their general practitioner about taking a VR test to assess their Alzheimer’s risk.
The story does note that previous work has been done determining that navigational skills are an early indicator of Alzheimer’s risk. However, the story makes it seem like the use of VR technology is what’s new here. That’s not true. We found studies using VR to assess issues related to navigation ability and Alzheimer’s dating back to at least 2008 and 2009. So, what sets the forthcoming study apart? We wish the story had made that clear.
We couldn’t find a related news release, so we’ll rate this N/A.
Systematic, Criteria-Driven
Why fact-checking is not enough to prevent the spread of misleading health care news...@HealthNewsRevu https://t.co/fzC2JbDYGV
Simply fact-checking an article "fails to capture questions of nuance and context that arise – or should arise – whenever evaluating medical evidence." https://t.co/HrYkGj12q1
We wrote only about efforts that label themselves as “fact-checking” projects. You’ll see that if you read the piece.
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like