Kathlyn Stone is an associate editor at HealthNewsReview.org and manages the news release review project. She tweets as @KatKStone.
We’re closing in on our 500th news release review since launching this service in April 2015. What have we learned? We’ve learned that overall, universities, academic medical centers, hospitals, trade groups and industry are committing many of the same mistakes and omitting the same key information they were in 2015 when we started. We’re dedicated to helping these organizations improve through the constructive criticism we provide in our reviews and our unprecedented offer to review news releases prior to publication.
The 169 news releases we reviewed in 2017 received the following star score ratings:
0 stars – 1 (0.6%)
1 star – 20 (12%)
2 stars – 63 (37%)
3 stars – 69 (41%)
4 stars – 16 (9%)
5 stars – 0 (0%)
The 0 – 5 star score is based on how many satisfactory ratings a release earns when judged on our 10 systematic criteria. The conversion to star score is explained here.
Why were there no 5-star reviews? Even high-quality 4-star releases commonly missed on some key criteria that prevented them from reaching the highest achievable score. These near-misses most often included failure to discuss cost and availability (rated “not satisfactory” on 90% and 61% of reviews in 2017, respectively), and to identify funding sources/conflicts of interest (rated not satisfactory on 49% of reviews).
Despite the generally low number of releases receiving high marks, there were some bright spots.
These four releases demonstrate how some writers addressed challenges that many others whiffed on.
Reviewers praised this release from the American College of Surgeons for “discussing surgical alternatives and costs, which were estimated to be $9,000 lower for the less invasive laparoscopic procedure.” Cost is the most often-missed criterion in our news release reviews. We offer a primer with tips and resources for writing about the costs of health care interventions.
Another often-ignored criterion — comparison with existing alternatives — did find its way into this already strong release by MD Anderson. Reviewers noted that the release compared proton therapy results with historical controls and detailed limitations of the comparison. Reviewers also noted that “The release does mention intensity-modulated radiation therapy (IMRT) at the end and states the historical control was chemotherapy with conventional radiation.” Although our reviewers had some concerns with how the alternatives were presented, the satisfactory rating demonstrates our willingness to give the benefit of the doubt on any reasonable attempt to address our criteria.
Our reviewers were especially impressed with the release’s description of diagnostic accuracy, which included discussion of a statistical analysis that measures tradeoffs between sensitivity and specificity. The release noted that conventional imaging combined with the new test had a value of 0.85 according to this measure, while conventional imaging alone had a value of 0.60. The release offers that “a value of 0.50 indicates that a diagnostic test is no better than chance,” to give readers a benchmark for comparison.
This release by Oxford University Press also received praise for describing the benefits of the intervention in clear, meaningful terms. The release explained that based on a study, the HPV screening test alone may be enough to detect the vast majority of cancers and so-called “pre-cancers” without the added second test known as the Pap test. Reviewers said: “We were especially pleased that the release included a clear description of the benefit using absolute, rather than just a relative, risk number….Besides this number, the release included specific numbers for benefits of each test.” Read more about the importance of reporting absolute risks here.
Our analysis of the year’s output suggests that top-scoring news releases have the following elements in common:
These attributes are generally all absent in lower quality news releases as demonstrated in the examples below.
Just one news release walked away with zero stars. A Kings College London release on a natural tooth repair method using an Alzheimer’s drug earned this review headline:
According to reviewers: “We rarely give 0 ‘stars’ for reviews but this is one situation where we had to.”
This release not only skipped any mention that the study was done only in mice, it also made an unjustifiable leap by suggesting that because the drug has “previously been used in clinical trials to treat neurological disorders including Alzheimer’s disease” that it would speed the pathway for a new dental treatment. The release had multiple other problems as well leaving readers with none of the information they deserved.
Reviewers pointed out that “…the release doesn’t acknowledge an important conflict of interest that is addressed in the study manuscript itself: “H.J.K. is the inventor of a patent related to the monitoring of endometrial glandular development.” When a study co-author is poised to make money off the intervention being touted in a news release, it’s a detail readers should be told. We’ve written plenty about conflicts of interest in health care and health care journalism this year.
Northwestern University wasn’t the only organization to misapply the term “breakthrough” in a news release headline. Far from it. But it’s a good example of how this term is casually slapped on almost any new medical technology these days. Our reviewers wrote: “Is it proper to call something a ‘new breakthrough’ when it has apparently been on the market for four years? And when its effectiveness compared with other approaches hasn’t been properly established?” Read about other words that writers should think twice about employing in health care news releases.
Here’s another release that didn’t warn readers that the research was conducted in mice, not people. The release made a troubling claim that components of the spices oregano and thyme may represent a “cure” for cachexia, the wasting disease experienced by cancer patients. Further, the university indicated it would soon be licensing the product.
When we contacted the University of Iowa about the problems with the news release, the response was insufficient, to put it mildly. We blogged about the University’s policies regarding conflicts of interest and its process for issuing news releases twice. Their lack of any substantive response is troubling and raises questions about the credibility of the institution’s health care-related communications.
While there’s a lot of crappy PR out there, we know there is also great work being done and we’re eager to shine a light on it. If you’ve written a 5-star news release or you see one that meets all or most of the criteria for a high quality, informative release, send it our way and we’ll consider reviewing it.
Please keep in mind that our criteria are geared to reviewing releases that focus on a health care intervention.The release must include a claim about a drug, medical device, procedure (such as a type of surgery), or a test, and it must be posted online on your website, or at one of the news release wire services such as EurekAlert! and Newswise. (Remember to check out our 10 review criteria before submitting the review for consideration.) Forward releases to feedback@healthnewsreview.org
—
More 2017 year-ender posts
Friday, December 15: Kicking off 2017 HealthNewsReview.org year-ender series
Tuesday, December 19: Ranking news organizations by star score. Who’s done best? Who has the most room for improvement in 2018?
Wednesday, December 20: A great year for research progress in mice – although news often made it sound like it was in humans.
Thursday, December 21: Perspectives from the woman who wakes up every morning and searches the news for us every day – all year long.
Friday, December 22: Memorable lines from memorable interviews from 12 podcasts we produced this year.
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Comments are closed.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like