Medicating kids with ADHD keeps them safer? Not so fast, Wall Street Journal

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

Costs of the drugs are not discussed, nor is whether they are generally covered by insurance, or how long kids would have to take them (for a few years? into adolescence? into adulthood?).

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story provides some percentages but no absolute numbers. It says:

The researchers found fewer and less severe injuries and health problems among the treated children: a 3.6% reduction in sexually transmitted infections; 5.8% fewer children who sought screening for sexually transmitted infections (suggesting they had had an unprotected sexual tryst); and 2% fewer teen pregnancies. That adds up to a lot fewer teenagers in trouble.

The Princeton news release about the study provides actual numbers related to outcomes, including injuries, which is missing from the Wall Street Journal story even though the story makes a claim about injuries being less likely.

In absolute numbers in a sample of about 14,000 teens diagnosed with ADHD, it translates into 512 fewer teens contracting an STD and 998 fewer having a substance abuse disorder. There also would be 6,122 fewer yearly injury cases for children and teens under 19 years old.

We would have liked to have seen both types of statistics included in the story.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

There is no mention of harms in the story. The adverse effects of the drugs in question would be key information–do they outweigh the benefits put forth in this story?

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story explains that the study is a retrospective review of data from Medicaid in one state. Readers need more details to ascertain how strong a conclusion can be made from this kind of data review. They should be told specifically that this kind of study can’t prove that medicating kids with ADHD “reduces risky behavior” nor “keeps them safer.” (This is an error also seen in the news release headline.)

The piece makes a reference to one other study, saying “randomized, controlled studies—experiments carefully designed to establish cause-and-effect relationships—have reached the same conclusion: that medication to control ADHD can reduce the high price in psychic pain,”

But the study that is linked to is not a randomized, controlled study. It, like the study in question, is a retrospective review of health records. Again, these don’t prove cause and effect.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

There are no independent sources in the piece, which straddles the line between commentary and news. We found it in the health section where it wasn’t labeled as opinion and the headline reads like a straight news story. But some first-person passages make one wonder if it’s more of an opinion piece.

Regardless, we would have liked to have seen at least some references to work outside of this one study, since the claim being made is significant, while the evidence being given for the claim is less so. We want to know: What kind of evidence is required before pediatricians can feel confident saying “treating ADHD will not only take care of immediate behaviors, it will create safer behavior as your child grows?”

Does the story compare the new approach with existing alternatives?

Not Satisfactory

There is no mention of alternatives in the piece, but there are several.

Does the story establish the true novelty of the approach?

Not Satisfactory

The piece does not establish the novelty of the findings. Again, this was something better handled by the news release, which states “While previous research has demonstrated the effectiveness of medications in treating the core symptoms of ADHD, little has been known about the effects of treatment on health, behavioral and educational outcomes in the long run. Evidence so far points to positive effects on some outcomes but not others.”