Stories that include claims about new technologies need to be more rigorous than this one was. Consumers don’t need a text book or a journal article. But our 10 criteria can be addressed readily. And if they were, consumers would be given far more of the information they need to make sense about new approaches and research studies.
This story missed on what are arguably our most two important criteria: explaining costs and benefits. These issues matter in all health care news stories.
It also missed a key point addressed in our “Does the story seem to grasp the quality of the evidence?” criterion. See below.
The story didn’t include anything about costs.
What will it cost to use the “computer-based imaging tool” described in the story? We’re not told. There is no information on the cost of expanding the use of retinal photographs to a wider population.
A clear example of how incomplete relative risk reduction/benefit figures can be. The story only says that “Those with the narrowest vessels at the beginning of the study were four times more likely to have developed glaucoma a decade later.”
But it does not explain:
Not applicable. There wasn’t any discussion of potential harms and we can’t think of any – besides the obvious one of unnecessary glaucoma screening in some cases.
At least the story included the independent perspective from an eye surgeon, who said:
“It remains to be seen if this approach will help us identify people at risk for glaucoma sooner,” he says. “We have a number of tools now to help us do that, but we’ve got to get people in our offices to use them.”
But there certainly wasn’t the kind of rigorous evaluation of the evidence that we would have liked to see in such a story.
The test is actually just computer analysis of retinal photographs, which many people have experience with for diabetic eye screening. Placing the study in that context would be very helpful for readers who are trying to understand the study. Also, the story doesn’t mention that 33% of participants are lost to follow-up. Since only about 3% of participants with complete follow-up developed glaucoma, the actual number of patients among the “missing” 33% who developed glaucoma are likely to significantly influence the results and potentially the conclusions to be drawn.
No disease mongering of glaucoma in the story.
The closing comment from an independent eye surgeon provided some necessary – albeit late and brief – perspective.
Again, we’ll give the story the benefit of the doubt for at least nodding in the direction of other alternatives. For example:
There was no discussion of what the “computer-based imaging tool” really was. Is it equipment already readily available and in use? If not, what is it, and what would it take to implement?
No details were provided – a weak spot in the story.
The story should have made the connection between retinal photography that is currently done for diabetic eye screening and this new, alternative use of the same images.
Related to the unsatisfactory “Availability” score above, since no details were given about the “computer-based imaging tool,” we don’t know anything about how novel it is (or isn’t).
It does not appear that the story relied solely or largely on a news release.
Systematic, Criteria-Driven
The tale of two “negative” studies. Or, how spin is applied to make research findings look more impressive. A must read. https://johnmandrola.substack.com/p/the-tale-of-two-negative-studies
The @nytimes chose to interview one of the editors of the @AnnalsofIM (publisher of this paper) who "gushed with extraordinary glee: It’s huge! There are very few things that reduce your mortality by 30%."
Turns out coffee is not one of those few things despite all that glee. https://twitter.com/garyschwitzer/status/1533202075928215558
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like