Health News Review

A paper published in Pediatrics, “Respiratory Tract Illnesses During the First Year of Life: Effect of Dog and Cat Contacts,” is getting lots of news attention, but most of it misses what most such stories usually miss:  You can’t prove cause-and-effect from an observational study.  And there are big limitations to research based on people keeping diaries and answering questionnaires.

The study concludes:

“…that dog contacts may have a protective effect on respiratory tract infections during the first year of life. Our findings support the theory that during the first year of life, animal contacts are important, possibly leading to better resistance to infectious respiratory illnesses during childhood”

The Wall Street Journal reported:

“While the study tracked just under 400 babies, the researchers said the results were statistically significant because it relied on weekly questionnaires filled out by parents.” (Our reaction:  Huh?  This sentence makes no sense.  Statistical significance is not determined by parents filling out questionnaires.)

WebMD reported:

“It’s not clear why living with a dog makes such a difference.”  (Our reaction:  It’s not clear that living with a dog DOES make a difference. Making a difference means you’ve proved cause and effect and this study didn’t do that.)

HealthDay reported:

“Exposure to cats also showed a protective effect, but it wasn’t as strong as the effect from dog exposure.” (Our reaction:  No protective effect was established in this observational study.) 

None of these prior news organizations, and none of the following mentioned the limitations of observational studies and that they can’t establish a causal relationship.  Not Reuters Health nor the Los Angeles Times, nor CNN. (Addendum 90 minutes later:  Nor CBS.  Nor TIME.com. Nor The Toronto Star. )

From our early morning sweep of the news, only a My HealthNewsDaily story on MSNBC.com included what we were looking for:

“The link between pets and fewer infections held even when researchers took into account factors known to affect infants’ infection rates, such as breast-feeding and number of siblings. Still, the researchers acknowledged that couldn’t account for all such factors, and noted that they found a correlation, not a cause-and-effect relationship.”

That wasn’t so difficult, was it?

Our usual reminder:  Journalists and consumers should read our primer, “Does the Language Fit the Evidence?  Association Versus Causation.

Comments

Steven J. Raphael posted on July 9, 2012 at 2:34 pm

Unfortunately, the WSJ article quote “makes sense” in that the measures of statistical significance used in the article refers to a measure of “diary weeks”. The child’s health was correlated with the categories of pet presence on a weekly basis. This is problematic statistically as it greatly reduces the real “significance” of the relationship for it to be in so few sources. The different families may be statistically independent but the same family over many weeks is not independent statistically. The sample does not appear to be sufficiently random for my tastes (they seemed to be excluding urban areas because pets may not go outside but there is no discussion of the biases of selecting non-urban people nor is there much thought about the confounding influence of choosing to own a pet).

I am not a clinician nor an immunologist but there seems to be issues on the medical side of this. If there is any relationship between pets and infant health then surely it happens over time (“building” immunity). The way the research is constructed seems to be saying the relationship is with immediate pet presence (like one week of the pet going outside is protective but if the pet is gone the next the situation is totally different) and does not analyze lags or steady-states.

    Gary Schwitzer posted on July 10, 2012 at 4:41 pm

    Steven,

    Yes, I think I know what the story was trying to convey. But the sentence in question made no sense as written and is a non sequitur without further explanation. Remember: this is written for a general consumer news audience. If you’re going to try to explain statistical significance, you’re going have to give it a better shot than this.

Donald Hackett posted on July 10, 2012 at 6:24 am

I believe dailyRx inserted appropriate disclaimers: http://goo.gl/6x0YD

Helen Branswell posted on July 10, 2012 at 9:07 am

I looked hard at this study, figuring it was going to garner a lot of coverage. But in the end I passed.

While I’m really wary of association studies, in this case that wasn’t the reason I chose not to write about this. It was because the claims (words) didn’t match the data (numbers).

Maybe I’m misreading the data, in which case this is going to be really embarrassing. But if I’m not, I can’t really see why this paper got published.

For instance, they say: “We showed that children who had dog contacts at home had less otitis and rhinitis and more healthy weeks than children without dog contacts at home…”

But when you look at the data in Tables 5, 6 and 7, you see that they rarely hit statistical significance for any of their analyses. Most of the confidence intervals straddle zero, few of the p values are below 0.05. The exception: The analysis for “healthy” — those data suggest a statistically significant positive association from dog and temporary cat exposure.

But as to the claim the study shows that kids with dog contact have less otitis and rhinitis, the data show the evidence for a positive association appears to only exist for otitis and then only when dogs were temporarily or often inside (Table 7) or inside <6 hr/day or 6-16 hr/day (Table 5). The only place they spotted anything statistically valid for rhinitis was in Table 7, for the analysis of when dogs were often inside.

They also say: "Cat ownership seemed to also have an overall protective effect, although weaker than dog ownership…." but according to the tables the only analysis that showed a statistically significant association was in Table 6, regarding the need for antibiotics. There it looks like there was an association between lower antibiotic use in children who lived in homes where a cat was temporarily indoors.

I couldn't really see that they found would they said they found.

Tara Haelle posted on July 10, 2012 at 12:25 pm

Hey Gary, I was paying attention at AHCJ12 this year ;) Check out my version on this study: http://bit.ly/PIcEQx

It was a tough one to write, but I believe I covered the bases that other outlets missed. (I’m still working on following Ivan’s advice in getting a biostatistician in my pocket because the data in this study was opaque in many areas.) I did not get an outside comment on it, so I’ll fail to meet the standards in that area, but I’d love to know if you feel I represented the study accurately otherwise.

    Gary Schwitzer posted on July 10, 2012 at 4:46 pm

    Tara,

    Yes, I know you were paying attention at our AHCJ12 workshop in Atlanta!

    You included this important sentence: “The study also cannot establish that having the animals causes the children to be healthier.”

    But whoever wrote the headline – probably not you, I know – was dead wrong: BABIES PROTECTED BY DOGS AND CATS. That definitive-sounding statement conveys causation. So it’s time for you to conduct some training internally for your headline writers.

      Tara Haelle posted on July 10, 2012 at 7:29 pm

      Thank you and ouch at the same time – you’re right. In this case (hiding head), I actually did write that headline. (I’ll take my lumps and kudos.) My attempt at brevity betrayed me – but I’m learning ;) And thanks for taking a look!

      Gary Schwitzer posted on July 11, 2012 at 7:28 am

      And I see that you’ve now changed the headline to “Dogs & Cats Linked to Healthier Babies,” which is closer to the truth of the statistical association that was reported.