This article consists overall of balanced reporting. There are a few missteps in presenting the quantitative evidence but some very strong points are present as well, including the acknowledgement that the studies under consideration are all observational in nature, and the corresponding reminder that “correlation does not imply causation.”
Retinopathy of prematurity (ROP) is a potentially blinding eye disorder. The article reports on a meta-analysis examining whether human breast milk, compared with formula, is associated with protection against developing ROP.
Cost is not a relevant criterion in this case as the comparison is made between breast milk and formula. Both feeding methods entail costs, both financial and other, that we feel are beyond the scope of the coverage.
There are a couple of improvements we recommend here. First, we wish that the drop in odds of ROP had been reported more precisely, as there are many different combinations of variables being compared in the study. To start, the disease ROP is separated into 1) any-stage and 2) severe. (These terms are used here and there in the article but not defined.) For each level of ROP, the study makes the following comparisons:
Out of the four comparisons, the drop in odds of ROP (any-stage or severe) of breast milk versus formula is the most eye-catching in the fourth comparison. This also happens to be the comparison reported in the second paragraph of the article. While we do not wish to presume this comparison was cherry-picked, we feel that if word count had been a factor in limiting the reporting of only one comparison above, the second comparison is the one that should have been reported. This is because the sample size is largest for the second comparison, giving more credence (or as statisticians might say, more power) to the numbers.
More precision is also warranted regarding the specific comparison being made. The statistics cited in the second paragraph fail to mention that the comparison is between exclusive human milk and exclusive formula. Later in the article, two more statistics, a 75% drop and 46% drop in the odds of ROP, are cited. Here, it would’ve been good to qualify the statistic by stating that these numbers relate to drops in any-stage ROP. Moreover, these are relative risk comparisons, and so it’s unclear if these impressive-sounding drops in risk translate to large absolute numbers. The story could have clarified that.
In the framework of our criteria, the intervention in this study is the feeding of human milk. The potential harms of human milk do not seem to warrant a discussion in this article however.
Sometimes statisticians sound like broken records with their “correlation is not causation” warnings. The article is one step ahead in this case. We find the following sentences that would make every statistician proud:
However, the new analysis only showed an association between breast milk and a reduced risk of ROP. It did not prove a cause-and-effect relationship due to the study’s design.
Also, the article shows a good grasp of the evidence quality by reporting some of the limitations of the study, including the limitation that the study looked only at a mother’s own breast milk, not donor milk.
The story safely clears our bar here, but it could have done even better to avoid phrases such as this one: “Breast-feeding a premature infant may help reduce the risk of a serious eye problem known as retinopathy of prematurity.” The active verb “help reduce” suggests a cause-and-effect relationship that, as the story tells us, is not appropriate to imply with a study such as this one.
We paused to take a second look at the statement that ROP “has become a leading cause of childhood blindness in recent time.”
That sounds scary, but the story qualifies things nicely when it notes, “The greater incidence of ROP is likely due to the increasing survival rate of very preterm babies in developed countries.” That’s good context because it explains that the condition is actually a consequence of a positive trend in health care.
Thumbs up here for interviewing an independent source, Dr. Adolfo Llanos. Although, we question the choice to end the article with the following quote from Dr. Llanos
The takeaway from this study, Llanos said, is to breast-feed as long as possible, if possible.
That’s slightly puzzling, since the stated conclusion in the original study is much more tentative
Based on current limited evidence, in very preterm newborns, human milk feeding potentially plays a protective role in preventing any-stage ROP and severe ROP.
The story would have probably done better to omit this source’s simplistic conclusion.
The article does not mention other preventative measures to take against ROP besides breast milk and formula. But neither does the original study, so we’ll leave this criterion as not applicable.
We won’t dock points from the story for not discussing the availability of formula and breast milk. Although those topics could certainly sustain such a discussion had the story chosen to address them.
The article clearly identifies the underlying study being based on a reanalysis of five previously published studies on ROP (or what is known in the lingo as a meta-analysis). The article might have indicated that this is one of the first such meta-analyses done on ROP.
We can be fairly confident that the article did not rely solely on a news release, especially since it has an independent source.