This news release about a smartphone-based infectious disease testing device confuses speed and device cost with benefits to patients and public health. It oversimplifies infectious disease detection and response. In their published study, the researchers reported an important achievement: comparable accuracy (under ideal conditions) with existing lab devices. However, the release doesn’t point out vital distinctions between performance in the lab and in real-world field use, or between the cost of a device and the total cost of a public health program that uses such a device.
The release does not put the status of this device into context with other similar devices being developed by universities and companies around the world. We note that the release does report the funding source and the commercial intentions of the researchers.
News releases should include essential context, not just certain facts that put researchers’ work in the most positive light. This release focuses narrowly on accuracy comparisons between an experimental device and existing lab equipment. It leaves reporters and other readers to discover on their own how this device compares to similar experimental products. The release highlights potential real-world health benefits without acknowledging the complex questions that need to be asked before anyone can know whether this device can deliver on the promises.
Infectious diseases are still a significant source of mortality and morbidity especially in low and middle income countries. A device that could improve the diagnosis of these diseases would be useful in these settings.
The news release reports that the device would probably cost less than $50 to manufacture. However, the cost of a lab device is only part of the cost of infectious disease testing. What sort of training is needed to accurately interpret results? Would the device be able to replace conventional lab tests or would it be used as a preliminary test, that would then be checked by conventional tests? It is possible that in real world circumstances, the device could lead many more people into testing, potentially increasing the total cost of testing programs. The health benefit could be worthwhile, but the news release should not claim that a device would reduce costs without evidence of total costs in real world use.
The release reports details of a direct comparison between test results produced by this device and standard lab equipment. However, relative accuracy of lab tests is not, by itself, a health benefit. The release says that using conventional lab testing services takes too long “by which time the infection may have become widespread.” The obvious implication is that this hand-held device would lead to more rapid and more effective outbreak detection in the field. Perhaps…but that’s not what was tested. The release should have been clearer that actual human health benefits won’t be known until after testing in the field.
Tests, even accurate ones, have downsides which were not mentioned in this release. The release stated that the device “provided false positives only about one percent of the time.” If field workers fanned out across a city and tested 100,000 people, that 1% false positive rate would mean that about a thousand people would be told they were infected when they actually were not. The release did not mention false negatives (telling an infected person that he or she is not infected).
The lead sentence of the release claims that this device “works nearly as well as clinical laboratories.” But what was actually reported was just a comparison of this device to standard lab equipment in an ideal laboratory setting as performed by the device’s developers. The release should have made clear that the device was not tested in the field or using people who had only the amount of training that would be available under actual use conditions. The release should have noted the limitations of the study. It would have been more appropriate for the release to state that the researchers made progress demonstrating the potential for their device but that the actual public health benefits won’t be known until after field testing.
We will give the release a satisfactory rating on this criterion. However, other than measles, it is not clear that other types of infections used in this study (including mumps, herpes and Lyme disease) are responsible for the type of fast-moving outbreaks that the release says this device could help identify more rapidly.
The release reports that the researchers have filed a patent and hope to commercialize their product. It also states that the work was supported by a Washington State University fund to support entrepreneurial endeavors.
As noted above, this test was done by the device’s developers under ideal lab conditions. The comparisons to existing devices and speculation about relative effectiveness should wait until field tests are done.
The release states that the researchers “hope to move forward with clinical trials that could lead to commercialization.”
Although the release does not claim this device is the first or most advanced of its kind, it conspicuously avoids mentioning similar devices. A quick search of PubMed.gov and other online sources reveals a number of reports on similar products being tested at competing institutions including UCLA and Macquarie University in Australia. The release does not explain how this device compares to others.
The release includes several speculative claims that this device could perform better than existing devices in reducing public health program costs, health care access and delivery, infectious disease control and more. None of these important goals has been tested.
Systematic, Criteria-Driven
The tale of two “negative” studies. Or, how spin is applied to make research findings look more impressive. A must read. https://johnmandrola.substack.com/p/the-tale-of-two-negative-studies
The @nytimes chose to interview one of the editors of the @AnnalsofIM (publisher of this paper) who "gushed with extraordinary glee: It’s huge! There are very few things that reduce your mortality by 30%."
Turns out coffee is not one of those few things despite all that glee. https://twitter.com/garyschwitzer/status/1533202075928215558
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like