Can a story get 4-5 stars on HealthNewsReview.org and still be badly flawed?

Posted By

Tags

Yes.

We had two this week:

And here are some examples from the archives:

Just as you can’t judge a book by its cover, you can’t judge a story by its star score or grade.  Please read our commentary.  We give reasons why we voted satisfactory or unsatisfactory on each of 10 criteria.  And there are times – as seen above – where stories address most or all of our criteria and still are lacking in key context, caveats or framing.

 

You might also like

Comments (6)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

David Finer

June 3, 2012 at 5:42 am

Hi Gary

I am glad you addressed the possible flaws not only in the media reports but I guess, in any rating system. I know you are constantly developing your thoughts on criteria, and that the German Mediendoktor.de, e.g. uses additional criteria to the ones on other Media Doctor sites and, indeed, Health New Review. Under the auspices of a research foundation, we have recently launched an MD-like service in Sweden on news about Complementaryt and Alternative medicine in the media. However, I envisage probably adding and revising rating criteria over time based on feedback from health care workers, journalists, laymen and others. Based on the story you penned above, on the limitations of the rating system as I read it, are you currently thinking of ways to include assessments of i.a. “key context, caveats or framing”? I appreciate the need to limit the number of criteria for the tool to be usable at the same time… but there are indeed also other important aspects of medical writing, some qwhich the current criteria do not

David Finer

June 3, 2012 at 5:42 am

Hi Gary

I am glad you addressed the possible flaws not only in the media reports but I guess, in any rating system. I know you are constantly developing your thoughts on criteria, and that the German Mediendoktor.de, e.g. uses additional criteria to the ones on other Media Doctor sites and, indeed, Health New Review. Under the auspices of a research foundation, we have recently launched an MD-like service in Sweden on news about Complementaryt and Alternative medicine in the media. However, I envisage probably adding and revising rating criteria over time based on feedback from health care workers, journalists, laymen and others. Based on the story you penned above, on the limitations of the rating system as I read it, are you currently thinking of ways to include assessments of i.a. “key context, caveats or framing”? I appreciate the need to limit the number of criteria for the tool to be usable at the same time… but there are indeed also other important aspects of medical writing, some qwhich the current criteria do not

David Finer

June 3, 2012 at 5:44 am

Sorry, the last words should read:

…but there are indeed also other important aspects of medical writing, some of which are quality criteria, which the current rating criteria do not address.

    Gary Schwitzer

    June 4, 2012 at 8:28 am

    David,

    Thanks for your note.

    We don’t post separate criteria for context, caveats or framing. But we embed our assessments for these qualities within our commentary on each criterion.

    It’s difficult to provide an overall grade for context; a story may provide ample contextual information on cost, for example, while providing no broader information on existing alternatives. Another story may address many criteria, but the framing strikes us as disease-mongering. We think that the more specific we can be, the more helpful is our critique.

    That is why we remind all readers of our site to go beyond the “grade” or “star score” and read our comments.

David Finer

June 3, 2012 at 5:44 am

Sorry, the last words should read:

…but there are indeed also other important aspects of medical writing, some of which are quality criteria, which the current rating criteria do not address.