This Reuters story draws readers in with news that scientists have developed a quick HIV test on a USB stick, which could be useful in the developing world.
While the story hints at the underlying method —a unique lab-on-a-chip test to estimate the amount of HIV in a person—it glazes over the study’s two-test design. And it’s a big miss: The story confuses the effectiveness of the traditional, test tube-based method with that of the experimental USB stick test that researchers hope will replace it, ultimately overstating the latter’s sensitivity. The reason this happened is because it seems to be based entirely on a news release, which also seems to misunderstand the study’s design.
News outlets have a responsibility to report accurately on medical research, and go a step beyond the news release. In this case, it appears most news organizations dropped the ball. These inaccurate news stories could very well stoke false hope among HIV field workers and their patients, and further erode public trust in news media.
The cost of either test in the study isn’t shared, and it’s a simple question to ask for a ballpark figure on what a mass-produced lab-on-a-chip might cost. Absent that number, a comprehensive HIV viral load test with traditional (and bulky) equipment is about $24.90 to $44.07. Such tests are recommended by the World Health Organization (WHO) 6 months after the start of anti-retroviral treatment, at 1 year, then every year after that.
The story doesn’t correctly report the sensitivity of the lab-on-a-chip method, and an Imperial College London (ICL) news release (which the Reuters story seemed to rely on heavily) gets this wrong, too. The key thing each piece missed: There were two tests. The study used a scaled-up and better-understood test tube assay (which indicates a positive through a fluorescence reaction) to validate and compare to the lab-on-a-chip test (which uses acidity/pH). The researchers ran biological samples through each test to cross-reference them.
Here’s how the numbers break down: The tube assay was 95% sensitive (meaning it missed 5% of HIV-positive samples) when more than 1,000 copies of HIV-specific RNA were present per milliliter — a limit WHO says is an acceptable sensitivity. Between 50-1,000 copies, the tube assay dropped to 88.75% sensitivity. (We’ll note here that some lab-grade tests can detect 100 times fewer copies with close to 100% sensitivity.) The pH-based lab-on-a-chip test, meanwhile, was 88.8% sensitive at more than 1,000 copies and 76.1% with 50-1,000 copies. That’s a far cry from 95% accurate.
The physical harms are pretty obvious, as the story’s comparison to a diabetes blood sugar test makes clear: Patients need to get a spot of blood onto the sensor. But the more important harm isn’t discussed here, where a simple “this isn’t ready for the field” statement would have covered. More than 11 in 100 HIV-positive people getting a false-negative test is not very comforting; they could be misled to think their anti-retroviral therapy is still working when it might not be.
As previously noted for our second criteria (quantifying benefits), the story didn’t correctly report the lab-on-a-chip’s “real-world” sensitivity, i.e. with human blood plasma. The assertion that 991 blood samples were run through the lab-on-a-chip test is wrong, too—only 164 were. The others were done with the non-electronic tube-based test. The two different tests aren’t mentioned, described, or distinguished in any way. It doesn’t appear the writer spoke with the researchers or read the abstract of the study, which summarizes all of these key points.
We didn’t find any outlandish descriptions of HIV or AIDS.
We did see any original quotes from an independent source, nor the study’s authors.
Readers are told “current tests to detect virus levels take at least three days and involve sending a blood sample to a laboratory,” which is basic but we’ll mark this one satisfactory. However, knowing how often the WHO recommends viral-load tests be taken, and how sensitive and specific they are, would have made this story stronger.
We’re told the technology is in the “early stages” and is thus not yet available.
The story notes the exploratory nature of the test and its speed, at least relative to a standard (though unnamed) lab test, hinting at its novelty.
Quotes were copy and pasted directly from the ICL press release, and we didn’t detect any signs of original reporting.
From the news release:
“However, monitoring viral load is crucial to the success of HIV treatment. At the moment, testing often requires costly and complex equipment that can take a couple of days to produce a result. We have taken the job done by this equipment, which is the size of a large photocopier, and shrunk it down to a USB chip.”
And the Reuters news story:
“Monitoring viral load is crucial to the success of HIV treatment. At the moment, testing often requires costly and complex equipment that can take a couple of days to produce a result,” said Graham Cooke, who co-led the research from the Imperial’s department of medicine. “We have taken the job done by this equipment, which is the size of a large photocopier, and shrunk it down to a USB chip.”