Via Imperial College London and DNA Electronics
This Reuters story draws readers in with news that scientists have developed a quick HIV test on a USB stick, which could be useful in the developing world.
While the story hints at the underlying method —a unique lab-on-a-chip test to estimate the amount of HIV in a person—it glazes over the study’s two-test design. And it’s a big miss: The story confuses the effectiveness of the traditional, test tube-based method with that of the experimental USB stick test that researchers hope will replace it, ultimately overstating the latter’s sensitivity. The reason this happened is because it seems to be based entirely on a news release, which also seems to misunderstand the study’s design.
But Reuters wasn’t alone; the Washington Post, International Business Times, Gizmodo, Live Science, and other outlets similarly relied on the press release and mischaracterized the study’s results.
News outlets have a responsibility to report accurately on medical research, and go a step beyond the news release. In this case, it appears most news organizations dropped the ball. These inaccurate news stories could very well stoke false hope among HIV field workers and their patients, and further erode public trust in news media.
The cost of either test in the study isn’t shared, and it’s a simple question to ask for a ballpark figure on what a mass-produced lab-on-a-chip might cost. Absent that number, a comprehensive HIV viral load test with traditional (and bulky) equipment is about $24.90 to $44.07. Such tests are recommended by the World Health Organization (WHO) 6 months after the start of anti-retroviral treatment, at 1 year, then every year after that.
The story doesn’t correctly report the sensitivity of the lab-on-a-chip method, and an Imperial College London (ICL) news release (which the Reuters story seemed to rely on heavily) gets this wrong, too. The key thing each piece missed: There were two tests. The study used a scaled-up and better-understood test tube assay (which indicates a positive through a fluorescence reaction) to validate and compare to the lab-on-a-chip test (which uses acidity/pH). The researchers ran biological samples through each test to cross-reference them.
Here’s how the numbers break down: The tube assay was 95% sensitive (meaning it missed 5% of HIV-positive samples) when more than 1,000 copies of HIV-specific RNA were present per milliliter — a limit WHO says is an acceptable sensitivity. Between 50-1,000 copies, the tube assay dropped to 88.75% sensitivity. (We’ll note here that some lab-grade tests can detect 100 times fewer copies with close to 100% sensitivity.) The pH-based lab-on-a-chip test, meanwhile, was 88.8% sensitive at more than 1,000 copies and 76.1% with 50-1,000 copies. That’s a far cry from 95% accurate.
The physical harms are pretty obvious, as the story’s comparison to a diabetes blood sugar test makes clear: Patients need to get a spot of blood onto the sensor. But the more important harm isn’t discussed here, where a simple “this isn’t ready for the field” statement would have covered. More than 11 in 100 HIV-positive people getting a false-negative test is not very comforting; they could be misled to think their anti-retroviral therapy is still working when it might not be.
As previously noted for our second criteria (quantifying benefits), the story didn’t correctly report the lab-on-a-chip’s “real-world” sensitivity, i.e. with human blood plasma. The assertion that 991 blood samples were run through the lab-on-a-chip test is wrong, too—only 164 were. The others were done with the non-electronic tube-based test. The two different tests aren’t mentioned, described, or distinguished in any way. It doesn’t appear the writer spoke with the researchers or read the abstract of the study, which summarizes all of these key points.
We didn’t find any outlandish descriptions of HIV or AIDS.
We did see any original quotes from an independent source, nor the study’s authors.
Readers are told “current tests to detect virus levels take at least three days and involve sending a blood sample to a laboratory,” which is basic but we’ll mark this one satisfactory. However, knowing how often the WHO recommends viral-load tests be taken, and how sensitive and specific they are, would have made this story stronger.
We’re told the technology is in the “early stages” and is thus not yet available.
The story notes the exploratory nature of the test and its speed, at least relative to a standard (though unnamed) lab test, hinting at its novelty.
Quotes were copy and pasted directly from the ICL press release, and we didn’t detect any signs of original reporting.
From the news release:
“However, monitoring viral load is crucial to the success of HIV treatment. At the moment, testing often requires costly and complex equipment that can take a couple of days to produce a result. We have taken the job done by this equipment, which is the size of a large photocopier, and shrunk it down to a USB chip.”
And the Reuters news story:
“Monitoring viral load is crucial to the success of HIV treatment. At the moment, testing often requires costly and complex equipment that can take a couple of days to produce a result,” said Graham Cooke, who co-led the research from the Imperial’s department of medicine. “We have taken the job done by this equipment, which is the size of a large photocopier, and shrunk it down to a USB chip.”
Systematic, Criteria-Driven
Monday at #ADA2021, an infectious disease epidemiologist, a journalist, and a researcher shared perspectives on communication of risk in the era of COVID-19: http://ow.ly/ROfR30rMeE7 @AstleyCM @garyschwitzer @berthahidalgo @ADA_DiabetesPro
Shoutouts to @susan_bewley @drkevinknopf @MichaelBaum11 in this call for more than fawning coverage from press releases for these kinds of stories.
Also, linked history inside goes back 5 years just on Grail apparently trying to become Holy Grail. https://twitter.com/garyschwitzer/status/1408849696416841731
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like