A pat on the back to Vox for excellent reporting on melatonin

Does the story adequately discuss the costs of the intervention?

Satisfactory

The story notes that melatonin can be bought in “a corner drug store…for just $6.” It is widely available, not only in drugstores, and is relatively inexpensive.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Satisfactory

The story offers quantified benefits in one place, citing a 2005 meta-analysis of 15 studies that reported “on average, melatonin decreased the time it took to fall asleep compared to placebo by 3.9 minutes, it increased the time spent asleep by 13.7 minutes, and increased sleep efficiency (a ratio of time spent lying [in] bed versus time sleeping) by 3.1 percent.”

However, the story also addresses benefits in non-quantified, but meaningful, ways. For example, the story notes that while other analyses of melatonin-sleep studies found that melatonin helps reduce the time it takes to fall asleep, “not all researchers agree that melatonin can make a meaningful difference” (the emphasis is in the story itself). The story also includes an entire section on the circumstances in which melatonin supplements may be more effective at helping people sleep — namely, when people are suffering from significant jet lag or have a melatonin deficiency. Though this section does not include any quantification of the potential benefit, it is useful and important context for consumers who may be considering taking melatonin for use as a sleep aid.

Does the story adequately explain/quantify the harms of the intervention?

Satisfactory

The story addresses harms directly, and in two ways. First, the story includes a section focusing specifically on side effects, such as drowsiness (presumably at undesired times) and headache. This section also notes that melatonin is “not harmless,” and can have more severe side effects in some users. Second, the story notes that taking large doses of melatonin can make the brain less responsive to the hormone.

However, it’s not clear from the story whether this could actually make a person’s sleep problems worse. Another point that would have been worth making is that melatonin can interact with other medications, including birth control pills, blood thinners and diabetes medications — making it worthwhile to check with a physician before determining whether to take melatonin. That said, this still earns a Satisfactory ranking.

Does the story seem to grasp the quality of the evidence?

Satisfactory

The story does a particularly good job here. For one thing, the story explains exactly why it can be difficult to compare different studies on melatonin and sleep, noting: “The studies that have been done on these questions have tested a wide range of doses on many different populations, which makes direct comparisons among studies very difficult, and limits what we can really learn from them.”

To address those limitations, therefore, the story looked at meta-analyses: papers that assess multiple studies simultaneously and use statistical tools to account for variation between studies. In fact, the story looked at five meta-analyses — four on melatonin and one on other sleep aids. And, for the four meta-analyses on melatonin, the story gives readers a concise overview of the type of studies included in each meta-analysis. The story also includes links to the relevant papers, which is always a good thing (if not always especially useful — depending on a given reader’s capacity for understanding the jargon of research articles).

Does the story commit disease-mongering?

Satisfactory

No disease mongering here.

Does the story use independent sources and identify conflicts of interest?

Satisfactory

The story cites four sources, one of whom is identified as an author of one of the meta-analyses. Well done.

Does the story compare the new approach with existing alternatives?

Satisfactory

The story does a good job here and — again — does so in more than one way. First, the story explicitly compares the quantified benefits found in the 2005 meta-analysis of melatonin-sleep studies (discussed above) to quantified benefits found in a 2007 meta-analysis of studies that focused on prescription sleeping pills. That’s extremely useful.

The story also notes the potential harms associated with prescription sleep aids, which is useful context. Second, and this could also have fit into the “benefits” section above, the story notes that melatonin — or any sleeping pill — may not address the myriad potential causes of sleep problems, ranging from stress and noisy neighborhoods to lack of exercise and drinking too much caffeine. We think it’s valuable to note that these environmental and lifestyle factors can play a key role in someone’s ability to get “good sleep.”

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

As noted above under “Costs,” the story makes clear that melatonin supplements are widely available.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The release does not appear to be based on any news release that we could find.