Starting with the headline of the article, so-called “night milk” (cow’s milk collected at night) is touted as a possible treatment for anxiety and insomnia. This is certainly the type of headline that grabs one’s attention. The claim is difficult to support, however, due to the inherent limitations of the underlying study. The article’s main shortcoming is that it mischaracterizes the quality of evidence presented by the study. Our concern is that the story misreports the comparisons that were made in the study. Contrary to what the article reports, night milk is never directly compared against day milk in the study. Nor is it ever directly compared against diapezam. To its credit, the article does a very good job in correctly recognizing that the underlying study only concerns animals and thus cannot be generalized to humans without further research.
The CDC states that, “getting sufficient sleep is not a luxury – it is a necessity – and should be thought of as a ‘vital sign’ of good health.” Further, more than one quarter of Americans claim to not get enough sleep and around 10% suffer from chronic insomnia. The sleep-promoting and anxiety-reducing effects of night milk are worth studying because the findings may suggest its potential as a natural aid for sleep and anxiety related disturbances.
Although the cost of night milk is not discussed directly, the reader is given a hyperlink to the company website of Milchkristalle GmbH, a producer of night milk, where the price of night milk may be found. (Though we find it strange that a hyperlink of night milk should direct the reader to a specific company selling a night milk product.)
The reader is left to ponder statements such as “mice that got night milk were significantly less active…” But how much less active?
The story does make a good faith effort to report some number-based outcomes from the study. It says, for example, “Balance and coordination were measured by the number of falls from a rotating bar during a 20-minute period: Mice fed night milk on average fell four to five times, about twice as often as mice given day milk. Diazepam-treated controls fell about nine times, while the water-fed controls fell twice.” But then again, it’s not exactly made clear how this result relates to the outcomes that readers are interested in such as sleep or anxiety. We’ll reluctantly rate this Not Satisfactory.
Drinking milk is common enough that it hardly warrants an explanation of the associated harms.
The story earns points for declaring up front that the study was conducted in animals and including a caveat at the end that the study can’t be generalized to humans. This is an excellent practice that helps counterbalance some significant concerns we had with other aspects of the piece.
Specifically, the statements
… the study found that milk collected at night, or night milk, had enhanced sedative effects in mice compared with milk produced during the day.
and
Night milk significantly decreased the rodents’ physical activity, balance and coordination and increased sleep time compared with day milk, the research showed.
are misguided. The underlying study never directly compares night milk to day milk. Rather, the treatment arms of 1) day milk at three doses, 2) night milk at three doses, and 3) diazepem are each compared to the treatment arm corresponding to the control group, which consists of mice who only received distilled water.
We think the confusion may stem from the fact that the study also reports one-way ANOVA results. (Just fancy statistics jargon for comparing treatment effects between multiple groups.) A significant ANOVA result allows one to conclude that not all of the treatment arms are the same, i.e. there exists at least a pair of comparisons amongst the treatment arms that are significantly different from each other. ANOVA used in this way simply tests an omnibus hypothesis that is not very interesting in its own right, in that it is not a procedure that identifies which two experimental arms are different.
We acknowledge that the study itself is not clear on this point, making it difficult for journalists to analyze the results. Inclusion of an independent source might have helped with the evaluation.
No independent sources are solicited. The only link included to an outside “source” is to the company that sells night milk.
The article states
Mice fed night milk were more inclined to explore open spaces, an indication of reduced anxiety that was comparable to the effects from consuming diazepam…
The design of the study, however, does not support this claim because a direct comparison is never made between night milk and diazepam. Since we’ve already addressed this issue above, however, we’ll give the story credit here.
The article makes an effort to establish the availability of night milk by providing a hyperlink to a company that sells night milk. But the website obviously caters to German customers and seems like it could be a headache for US consumers to navigate. And we couldn’t find any other online sources of night milk besides this company that would be more accessible.
The article does not establish the context of the study adequately, making it hard to judge the novelty of using night milk to treat anxiety and insomnia. Is this the first such study to be conducted? This is where an independent expert could’ve helped out.
We couldn’t find a news release to compare with the story. But since no independent sources are quoted, we can’t be sure the story didn’t rely on a news release. We’ll rate this Not Applicable.
Systematic, Criteria-Driven
With the discontinuation of @HealthNewsRevu, there's a genuine concern that well-written and researched "reality checks" in healthcare are not economically interesting. This should be disconcerting to anyone, and yet I'm only seeing more and more piling into "promising findings"
Thanks for ALL you do and have done, Gary. https://t.co/93K4hzQXZj
Thanks for sharing this with all of us today, @garyschwitzer @HealthNewsRevu
A nice gift.
#healthcare #healthcarenews #journalism #reporting #factsmatter #truthmatters https://t.co/hNHdZDl2rc
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like