Journalists and consumers who rely on information presented in talks at scientific meetings need to realize how incomplete that information may be.
Dartmouth and VA researchers Steve Woloshin and Lisa Schwartz wrote an article in the Journal of the American Medical Association called “Media Coverage of Scientific Meetings: Too Much, Too Soon?” In it, they show how many news stories in one analysis reported on various types of abstracts, or short summaries of research, at scientific meetings. They reported:
“We found that research abstracts presented at prominent scientific meetings often receive substantial attention in the news media. This prepublication dissemination of medical research often brings findings to the public before the validity and importance of the work has been established in the scientific community. Adding to this concern, many of the abstracts receiving media attention have weak designs, are small, or are based on animal or laboratory studies; 25% remained unpublished more than 3 years after the meeting. Interestingly, presentations that receive front-page coverage are no more likely to be published than abstracts receiving less prominent coverage.”
The authors conclude:
“In addition, news organizations might also consider raising their threshold for reporting on scientific meeting abstracts at all. If they do choose to report on such presentations, they might make a concerted effort to emphasize the preliminary nature of data presented, and apply the same level of skepticism in covering these stories that they do in reporting on political matters. In this way, the press might help readers to develop a healthy skepticism about the breakthroughs they repeatedly encounter in the news. Scientists presenting at meetings can also help by routinely emphasizing the limitations of their work when interviewed by the press.
The current press coverage of scientific meetings may be characterized as ‘too much, too soon.’ Results are frequently presented to the public as scientifically sound evidence rather than as preliminary findings with still uncertain validity. With some effort on the part of meeting organizers, journalists, and scientists, it will be possible to better serve the public.”
Woloshin and Schwartz recommend the following language for journalists who choose to write about preliminary unpublished research presented at meetings:
“The findings presented are a work in progress. Because the findings have not undergone final peer review, they have yet to be independently vetted, and may change.”
We hope you can see why publication in a peer-reviewed medical journal – although no guarantee of integrity or quality – is an important screen. At least other scientists can evaluate and comment on the research methods and on the strength of the evidence . something not done at many scientific meetings.
Woloshin and Schwartz have written related pieces:
The Journal of the National Cancer Institute (“What’s the Rush? The Dissemination and Adoption of Preliminary Research Results“). Excerpt: “Physicians are confronted with preliminary research findings all the time. To decide whether the findings are good enough to change practice, they must be able to answer some fundamental questions. The most basic question, of course, is what is the rush?”
The Medical Journal of Australia (“Media reporting on research presented at scientific meetings: more caution needed“). Excerpt: “Work presented at scientific meetings is generally not ready for public consumption: results change, fatal problems emerge, and hypotheses fail to pan out.”