Read Original Story

Vox notes limitations of meta-analysis on starting school later, albeit not until end of story


4 Star


Starting school later might really help sleep-deprived teens

Our Review Summary

Homework is boringIt is clear that too little sleep is bad for humans, and health officials have long recommended that teenagers get eight to nine hours of sleep a night, a suggestion made difficult by early start times (often before 8 a.m.) in America’s schools. But how good is the evidence for the effects of later high school start times on students? A group of researchers conducted a meta-analysis and concluded that the evidence is not very strong. Among the 287 relevant studies found, only 18 (6%) were of good enough to quality for inclusion, and while those studies yielded some evidence for beneficial effects (later start times do seem to result in longer periods of sleep, for example), the researchers had a hard time finding links between later start times and both academic performance and behavioral issues.

The overall evidence, they concluded, is just too weak. Initially, the Vox story seems to ignore this take-away, especially with the headline that starting school later “might really help.” Later, the story does go on to explain some of the limitations of existing studies and includes the researchers’ call for more rigorous data.  Whether those cautionary comments, sitting at the end of the story, will help readers evaluate these results is hard to say.


Why This Matters

America’s schools are under pressure to establish later start times, a change that will be complex and potentially expensive to implement.  Such a modification needs to be grounded in good evidence that teenagers will, in fact, benefit from it. News stories must be clear on how conclusive the evidence really is.


Does the story adequately discuss the costs of the intervention?


Although specific, numerical costs are not mentioned, the story does a good job of reflecting on the kinds of costs that school districts would incur if they went to a later start time.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?


The story is specific about the average sleep advantages of a later start time among teenage participants in the 18 studies evaluated. For this reason, it rates satisfactory.

However, its discussion of other benefits—including fewer accidents involving teens behind the wheels of cars, mental health indices and actual school performance—is general and, with respect to school performance, more optimistic than the meta-analysis that is the centerpiece of the story.

Does the story adequately explain/quantify the harms of the intervention?


There are some drawbacks associated with delaying school start times, and the story does a good job of discussing them.

Does the story seem to grasp the quality of the evidence?


The story does a good job describing the meta-analysis of studies on later school start times, listing some of the limitations. It also lets us know more work needs to be done on the subject.

Does the story commit disease-mongering?


Sleep deprivation, while not a disease per se, makes it hard to function and may have long-term impacts, as well.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

No sources appear to have been interviewed for this story, though it contains a lot of citations to studies and quotes used in past stories.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

There are alternative public health measures that might get teens to sleep more (for example, a public education campaign). This context would have been helpful: What are the other options, and how effective might they be?

Does the story establish the availability of the treatment/test/product/procedure?


The story is clear that few schools have modified their start times in response to this issue. A map is a great addition in this regard.

Does the story establish the true novelty of the approach?


The reader will learn that the effects of eight or nine hours of sleep a night on teenagers have been demonstrated in other studies. What is novel is the meta-analysis’ reflections on the big picture, which suggests caution in making school policy changes on the basis of extant studies.

Does the story appear to rely solely or largely on a news release?


The story does not rely on the news release.

Total Score: 8 of 10 Satisfactory


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.