Another news story about the limitations of some studies

Yesterday we profiled a Wall Street Journal column about the statistical flaws in some studies. Today we point out a Los Angeles Times column that gives readers a better understanding of the strengths and weaknesses of epidemiologic studies. Excerpts:

“(Critics say that) far too many of these epidemiological studies — in which the habits and other factors of large populations of people are tracked, sometimes for years — are wrong and should be ignored.

In fact, some of these critics say, more than half of all epidemiological studies are incorrect.

The studies can be influential. Often, in response to them, members of the public will go out and dose themselves with this vitamin or that foodstuff.

And the studies also influence medical practice — doctors, the critics note, encouraged women to take hormones after menopause long before their effects were tested in randomized clinical trials, the gold standard of medical research.

Some of epidemiology’s critics are calling for stricter standards before such studies get reported in medical journals or in the popular press.

(One) of the foremost critics argues that epidemiological studies are so often wrong that they are coming close to being worthless. “We spend a lot of money and we could make claims just as valid as a random number generator,” he says.

Epidemiology’s defenders say such criticisms are hugely overblown.

They are “quite simplistic and exaggerated,” says Dr. Meir Stampfer, a professor of epidemiology and nutrition at the Harvard School of Public Health and a professor of medicine at Harvard Medical School.

…The debate is unlikely to be resolved any time soon. “If you put five epidemiologists and five statisticians in a room and have this debate,” (one critic) says, “and try to get each one to convince the other side, at the end of the day it will still be five to five.”

The important thing for journalists and for readers to understand is that there is a hierarchy of evidence – and that not all studies hold equal weight or power to point to a conclusion. Both the Wall Street Journal and Los Angeles Times stories are important in reminding us of that.