Read Original Release

Smartphone device’s early lab results no substitute for real-world infectious disease detection

Researchers use smart phone to make a faster infection detector

Our Review Summary

This news release about a smartphone-based infectious disease testing device confuses speed and device cost with benefits to patients and public health. It oversimplifies infectious disease detection and response. In their published study, the researchers reported an important achievement: comparable accuracy (under ideal conditions) with existing lab devices. However, the release doesn’t point out vital distinctions between performance in the lab and in real-world field use, or between the cost of a device and the total cost of a public health program that uses such a device.

The release does not put the status of this device into context with other similar devices being developed by universities and companies around the world. We note that the release does report the funding source and the commercial intentions of the researchers.

 

Why This Matters

News releases should include essential context, not just certain facts that put researchers’ work in the most positive light. This release focuses narrowly on accuracy comparisons between an experimental device and existing lab equipment. It leaves reporters and other readers to discover on their own how this device compares to similar experimental products. The release highlights potential real-world health benefits without acknowledging the complex questions that need to be asked before anyone can know whether this device can deliver on the promises.

Infectious diseases are still a significant source of mortality and morbidity especially in low and middle income countries. A device that could improve the diagnosis of these diseases would be useful in these settings.

Criteria

Does the news release adequately discuss the costs of the intervention?

Not Satisfactory

The news release reports that the device would probably cost less than $50 to manufacture. However, the cost of a lab device is only part of the cost of infectious disease testing. What sort of training is needed to accurately interpret results? Would the device be able to replace conventional lab tests or would it be used as a preliminary test, that would then be checked by conventional tests? It is possible that in real world circumstances, the device could lead many more people into testing, potentially increasing the total cost of testing programs. The health benefit could be worthwhile, but the news release should not claim that a device would reduce costs without evidence of total costs in real world use.

Does the news release adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The release reports details of a direct comparison between test results produced by this device and standard lab equipment. However, relative accuracy of lab tests is not, by itself, a health benefit. The release says that using conventional lab testing services takes too long “by which time the infection may have become widespread.” The obvious implication is that this hand-held device would lead to more rapid and more effective outbreak detection in the field. Perhaps…but that’s not what was tested. The release should have been clearer that actual human health benefits won’t be known until after testing in the field.

Does the news release adequately explain/quantify the harms of the intervention?

Not Satisfactory

Tests, even accurate ones, have downsides which were not mentioned in this release. The release stated that the device “provided false positives only about one percent of the time.” If field workers fanned out across a city and tested 100,000 people, that 1% false positive rate would mean that about a thousand people would be told they were infected when they actually were not. The release did not mention false negatives (telling an infected person that he or she is not infected).

Does the news release seem to grasp the quality of the evidence?

Not Satisfactory

The lead sentence of the release claims that this device “works nearly as well as clinical laboratories.” But what was actually reported was just a comparison of this device to standard lab equipment in an ideal laboratory setting as performed by the device’s developers. The release should have made clear that the device was not tested in the field or using people who had only the amount of training that would be available under actual use conditions. The release should have noted the limitations of the study. It would have been more appropriate for the release to state that the researchers made progress demonstrating the potential for their device but that the actual public health benefits won’t be known until after field testing.

Does the news release commit disease-mongering?

Satisfactory

We will give the release a satisfactory rating on this criterion. However, other than measles, it is not clear that other types of infections used in this study (including mumps, herpes and Lyme disease) are responsible for the type of fast-moving outbreaks that the release says this device could help identify more rapidly.

Does the news release identify funding sources & disclose conflicts of interest?

Satisfactory

The release reports that the researchers have filed a patent and hope to commercialize their product. It also states that the work was supported by a Washington State University fund to support entrepreneurial endeavors.

Does the news release compare the new approach with existing alternatives?

Not Satisfactory

As noted above, this test was done by the device’s developers under ideal lab conditions. The comparisons to existing devices and speculation about relative effectiveness should wait until field tests are done.

Does the news release establish the availability of the treatment/test/product/procedure?

Satisfactory

The release states that the researchers “hope to move forward with clinical trials that could lead to commercialization.”

Does the news release establish the true novelty of the approach?

Not Satisfactory

Although the release does not claim this device is the first or most advanced of its kind, it conspicuously avoids mentioning similar devices. A quick search of PubMed.gov and other online sources reveals a number of reports on similar products being tested at competing institutions including UCLA and Macquarie University in Australia. The release does not explain how this device compares to others.

Does the news release include unjustifiable, sensational language, including in the quotes of researchers?

Not Satisfactory

The release includes several speculative claims that this device could perform better than existing devices in reducing public health program costs, health care access and delivery, infectious disease control and more. None of these important goals has been tested.

Total Score: 3 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.