NOTE TO READERS: When this project lost substantial funding at the end of 2018, I lost the ability to continue publishing criteria-driven news story reviews and PR news release reviews - once the bread-and-butter of the site going back to 2006. The 3,200 archived reviews, while still educational, are getting old and difficult for me to technically maintain on the back end of the website. So I am announcing that I plan to remove these reviews from the site by April 1, 2021. The blog and the toolkit - two of the most popular features on the site - will remain. If you wish to peruse the reviews before they disappear, please do so by the end of March 2021. After that date you may still be able to access them via the Internet Archive Wayback Machine - https://archive.org/web/.
Read Original Story

Study links ADHD medicine with better test scores

Rating

4 Star

Study links ADHD medicine with better test scores

Our Review Summary

This story on the academic performance of kids with ADHD conveys an impression that is accurate in a very broad way: Elementary school kids who took ADHD meds did better on standardized tests than similar kids who didn’t take meds.

It makes clear that drugs remain one option among several and that behavioral therapy is an important alternative or complement to medication. 

It explains that this study is unique because it measures academic achievement not by grading, which is subjective and may be influenced by classroom behavior, but by standardized tests.  It puts these findings in a bit of context of similar research.

It uses an anecdote to provide detail about a typical experience, rather than an outlier patient. 

Having said that, the story fails to convey essential caveats: That the improvement was quite small, and that due to limitations of the design we don’t know which medications the kids were taking, how severe their condition was, and what other treatments individual kids got [or didn’t] get for their condition. And that for these reasons, the findings are interesting and incremental, but ought not to drive treatment decisions.

The report also fails to report the costs of drugs and non-drug treatment. As has often been mentioned in these reviews, during a time when so much attention is being paid to the cost of care vs. benefits, it is almost always worth including this information, especially about a treatment whose benefits are being discussed. 

Another significant shortcoming is a failure to acknowledge side effects of ADHD meds more fully. Any parent considering medications for ADHD needs to view what might be a minor improvement in academic achievement against risks, which rarely can be serious. Readers should have been told more about these risks. 

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The story does not mention the costs of ADHD medications. While some drugs are generic and cheap, particularly stimulants, other newer ones are more expensive.

The story should have mentioned costs, as a service to consumers and to those pondering the public health costs of various medical interventions in big populations.

The story should have reported the price for behavioral therapy, too, as well as non-medical responses such as tutoring, organizational support, etc.  

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Satisfactory

The story quantifies the findings two ways: in terms of increase in "points" on standardized tests in math [3] and reading [5], and how that translates into months of schooling [3 months in reading, 2 months in math].

This is good as far as it goes. 

But the story should have said that over the course of the six years the subjects were studied, these gains are quite small. 

It also should have been more precise with the word "points." We assume this means points on a 100-point scale, but this is not specified.

That all said, the story meets the minimum standard for this criterion. 

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The story mentions the most common side effects of ADHD drugs that concern parents, which include decreased appetite, weight loss and insomnia.

But the story fails to mention the more serious rare side effects, which include increased heart rate and blood pressure, breathing problems, tremors, tics, mood swings and [at the very worst extreme] cardiac problems, confusion, paranoia and  hallucinations.

And that’s just for the stimulants.

Side effects of Strattera, for example, include nausea, vomiting and abdominal pain. Strattera carries a "black box" warning related to suicide risk.

This is serious stuff. It should have been mentioned in a story whose conclusion is that the drugs have a significant benefit. 

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story describes the size, type, duration and study group adequately, but it misses some very important caveats:

  • That the study was conducted from 1999 to 2005, meaning findings would not apply to drugs introduced since then
  • The findings do not distinguish between children with different degrees of ADHD, co-morbidities or additional academic performance issues like learning disabilities. 
  • Because this is not a prospective, placebo-controlled clinical trial, it falls far short of establishing cause and effect.
  • The study’s findings looked at children from kindergarten through 5th grade, so they do not speak to the question of long-term academic achievement.
  • The study did not measure compliance
  • Any observational study has the inherent pitfall of confounding: the parents who choose medication for their kids may be engaged in a way that increases odds of higher academic performance, for instance. 

These weaknesses in the study itself would not disqualify the study for coverage. But the report should have referred briefly to at least a few–or at least offered a broader, single-sentence caveat about the methodology.

Does the story commit disease-mongering?

Satisfactory

The story does not exaggerate the prevalence or severity of ADHD.

The anecdote at the end is a good example of using a personal story that illustrates a typical patient rather than an outlier. It adds useful context about real-life treatment experience. 

Does the story use independent sources and identify conflicts of interest?

Satisfactory

The report includes the following sources. This is adequate sourcing for a story of this type and size:

  • The lead researcher
  • The Academy of Pediatrics treatment guideline
  • Two credible independent physicians not affiliated with the study
  • A patient with ADHD who has a long treatment history

Does the story compare the new approach with existing alternatives?

Satisfactory

The story mentions several treatment options, including behavioral therapy, which one source says should be the first-line treatment. The story repeatedly describes medication as one option, not the only one. 

The useful anecdote at the end also suggests that counseling, tutoring and exercise can help manage the condition. 

It would have been useful to say that no treatment at all may be the most common response, sometimes done carelessly but other times as a decision.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

No specific ADHD drug is mentioned, but the report says that several drugs were used by study participants, mostly stimulants. The story states that about half of young people with ADHD take some type of medication, which implies wide availability. 

 

Does the story establish the true novelty of the approach?

Satisfactory

The story states that this was "the first large, long-term study suggesting this kind of benefit from the widely used drugs."  

Does the story appear to rely solely or largely on a news release?

Satisfactory

Because of the number of sources cited – some independent of the study – it’s clear that the story didn’t rely solely or largely on a news release.

Total Score: 7 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.