Posted by Gary Schwitzer in Limits of observational studies
A paper published in Pediatrics, “Respiratory Tract Illnesses During the First Year of Life: Effect of Dog and Cat Contacts,” is getting lots of news attention, but most of it misses what most such stories usually miss: You can’t prove cause-and-effect from an observational study. And there are big limitations to research based on people keeping diaries and answering questionnaires.
The study concludes:
“…that dog contacts may have a protective effect on respiratory tract infections during the first year of life. Our findings support the theory that during the first year of life, animal contacts are important, possibly leading to better resistance to infectious respiratory illnesses during childhood”
“While the study tracked just under 400 babies, the researchers said the results were statistically significant because it relied on weekly questionnaires filled out by parents.” (Our reaction: Huh? This sentence makes no sense. Statistical significance is not determined by parents filling out questionnaires.)
“It’s not clear why living with a dog makes such a difference.” (Our reaction: It’s not clear that living with a dog DOES make a difference. Making a difference means you’ve proved cause and effect and this study didn’t do that.)
“Exposure to cats also showed a protective effect, but it wasn’t as strong as the effect from dog exposure.” (Our reaction: No protective effect was established in this observational study.)
None of these prior news organizations, and none of the following mentioned the limitations of observational studies and that they can’t establish a causal relationship. Not Reuters Health nor the Los Angeles Times, nor CNN. (Addendum 90 minutes later: Nor CBS. Nor TIME.com. Nor The Toronto Star. )
From our early morning sweep of the news, only a My HealthNewsDaily story on MSNBC.com included what we were looking for:
“The link between pets and fewer infections held even when researchers took into account factors known to affect infants’ infection rates, such as breast-feeding and number of siblings. Still, the researchers acknowledged that couldn’t account for all such factors, and noted that they found a correlation, not a cause-and-effect relationship.”
That wasn’t so difficult, was it?
Our usual reminder: Journalists and consumers should read our primer, “Does the Language Fit the Evidence? Association Versus Causation.“