TIME’s ‘pink noise’ sleep machine story gives too-rosy account of benefits

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The article makes clear that commercial development of a pink noise generator is in the future, so it’s too early to say what such a technology would cost. Indeed, we’re told that the researchers hope to develop an affordable device. Affordable is too vague to be helpful here. We encourage reporters to at least dig up some numbers on existing alternatives. For instance, there are 99-cent apps to generate white noise and presumably higher quality speaker devices that run up to $120. But the pink noise study relied on timing the noise to sleep waves — so would a commercial product need to include an electroencephalogram device as well? That seems a different price category altogether.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story tells us slow wave oscillations increase, but not by how much, or whether the increase is significant. The article is lacking important details around the intervention such as what an increase of slow wave oscillations might mean, say to a sleep wave expert or to an individual with sleep problems. Readers also learn that in a memory test taken in the morning, participants did three times better. But there are no details on what the memory test is or even what the “three times better” is compared to. (To the memory test the evening before? Or to the morning memory test after the no pink noise night?)

The absolute number (exact number of patients seeing a measured benefit) would be preferable to the relative number (“performed three times better”), particularly in a small study like this one, with only 13 participants.

In addition, there’s no discussion of how a memory test after sleeping can relate to long-term term memory effects. Improving memory for one day is not likely to be meaningful clinically.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The article does not mention harms at all. There may be no real harm to hearing these sounds as one sleeps, but we’d like to see that question addressed.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The article makes clear it’s a small study and the study author says that bigger and longer term studies are necessary. That’s good, but we’d like to see readers warned a bit more explicitly that these data are not nearly enough to make any kind of conclusion. And the story’s headline claims — unequivocally —  that pink noise does indeed improve sleep and memory, which is an overreach.

The outcome (a “memory” test) really doesn’t seem like a rigorous outcome and certainly not one that can be extrapolated to long-term memory effects.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The only source quoted in the article is a study author. The article states that one of the study authors (it’s not clear whether it’s the one quoted or not) has co-founded a company to develop a commercial device.

We applaud the story for including this conflict of interest but we’d like to see at least one source not involved in the study for an unbiased perspective.