CBS News speculates about what at-home sperm-checking device ‘could’ do

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

Only part of the prototype’s projected cost is discussed: $5 for a microchip device. While this sounds good, the cost of the “tiny weight scale” isn’t mentioned, nor is the “optical attachment” that could be 3D-printed. However, a consumer 3D printer can cost $1,000 or more, not including supplies.

Readers also are told male fertility screening is costly, but we’re never told what the cost is.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

Readers are told each test is 98% accurate, but aren’t told where that number comes from or what it means.

Equally as important, the story doesn’t explain that sperm count is only one facet of fertility, at least as defined by the World Health Organization; motility, morphology, pH, and other factors are similarly important, though less frequent issues, and require more advanced testing to establish.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Potential harms aren’t discussed, and there’s a significant one: a man receiving a false-negative test (he believes his sperm count is too low, when it’s actually normal) or a false-positive (his sperm count is low, but a test tells him it’s normal) — and telling his partner this incorrect information. For a couple that’s trying desperately to conceive, that could be crushing news or deceptive false-hope. Any news story about a new screening test should discuss the false-negative and false-positive rates.

For example, a story at LiveScience captures these well, telling readers that about 98% of people with no training could detect abnormal sperm samples from a fertility clinic with the new device. The sensitivity of the test’s ability to actually detect a sample with a low sperm count was 99.3%, but only 89.4% of normal-sperm-count samples were correctly read by the device. This means more than 10% of men with a normal sperm count might be lead to believe their numbers are abnormal.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

There’s mention of the device being a prototype–and a quick mention that the researchers are “refining” the test before seeking FDA approval. Yet other than that, there are few details to help readers discern the quality of the evidence. For example what needs to be refined before FDA approval? What were the limitations of this current study?

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

No independent sources were included, and the story didn’t disclose that the one source who was interviewed–researcher Hadi Shafiee–holds a patent on the technology.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

Readers are not given any details on the obvious alternative–going to a clinic for testing, or using one of the other already-available at-home devices. These and other details aren’t addressed, and they’re important in establishing the significance of the new prototype device.

Does the story establish the true novelty of the approach?

Not Satisfactory

The story takes the stance that this device is superior to others, stating the new prototype “could eventually shake up the world of fertility testing by allowing men to evaluate their sperm in their own homes.” However, at least two at-home, relatively low-cost, and FDA-approved test kits are already on the market. SpermCheck is a simple chemical test that costs about $40-$50, for example, and The Trak is physical test that uses a miniature centrifuge and retails for about $200. Both achieve similar or greater reliability as the prototype described in the story.

And while the novelty of using a smartphone may be an attractive bell-and-whistle for some men, it’s not necessarily unique. For instance, The Trak has an app — though it doesn’t directly measure samples. (It instead functions as a log for tests and lifestyle guide to improve sperm count and help men conceive with their partners.)

The real novelty, as LiveScience’s story points out, is that smaller fertility clinics can’t afford semen analysis devices, which can cost between $50,000 and $100,000. They could, however, afford a smartphone-assisted device.