November 23, 2010

More questions about peer review: preference for splashy findings

Posted By

Tags

Publication bias in medical journals is a well-known phenomenon. The latest evidence is explained in Frederik Joelving’s intriguing piece for Reuters Health, “Journal experts prefer ‘splashy’ findings.” Excerpts:

Given similar, fabricated reports that compared two treatments, reviewers from a pair of top-line orthopedic journals recommended publishing the fake results in 97 percent of cases when there was a difference between treatments, but only in 80 percent of cases when there was no difference.

“No-difference studies affect practice just as much as positive ones, but they aren’t as sexy,” said Dr. Seth Leopold of the University of Washington in Seattle, who led the new research. “Something splashy, something new, is more exciting to everybody.”

The problem is that favoring studies that identify, say, a new drug as superior to an older one will make the newer drug seem better than it really is.

As a consequence, when doctors scour the scientific literature about a given treatment, they don’t see the whole picture.
…
“We felt that these results really confirm beyond any reasonable doubt that positive bias occurs at the level of peer review. That is critically important for the integrity of the medical literature,” he added.

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Comments are closed.

COMMENTS button We welcome comments, which users can leave at the end of any of our systematic story reviews or at the end of any of our blog posts.

But before leaving a comment, please review these notes about our policy.

You are responsible for any comments you leave on this site.

This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.

We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.

We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.

There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”

The authors of the Retraction Watch comments policy urge commenters:

“Shed light, not just heat. Facts, challenges, disagreements, corrections — those are all fine. Attacking the person, instead of the idea or the interpretation, is neither acceptable nor helpful.”

We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.

And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.

The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.