Read Original Story

Guardian calls statistically insignificant study results a ‘breakthrough’ in care of children with autism


3 Star



Study offers potential breakthrough in care of children with autism

Our Review Summary

Pensive Beautiful little girlThis article in The Guardian reports on a study showing that parents who undergo a training program designed to help them pay rigorous and constant attention to their pre-school autistic children’s communications cues consistently improves the children’s symptoms and behavior.

According to the article, the success of the Pre-school Autism Communication Trial (PACT), conducted by a team of British investigators “surprised even the researchers” who designed the six-year-long, randomized study of 152 children ages two to four with so-called “core” symptom  severe autism. It reports that this “new form” of therapy, one that is parent-child driven rather than therapist-child driven, is a “potential breakthrough” in efforts to moderate the repetitive behaviors, tantrums, outbursts and other core symptoms, and that all of the children whose parents got the training showed significant and measurable improvement at the end of the one-year intervention.

What the article fails to say, however, is that the main outcome measurement — improvement long term at a follow up evaluation after five years — did not show a statistically significant difference between the intervention group and a group whose parents got conventional “parent education.” The intervention group had better results than the control group immediately after the initial treatment period, but both groups had worsened somewhat at the long-term follow-up assessment. Although the intervention group results were still better, this difference could have been due to chance. It’s uncertain, therefore, whether the intervention clearly improves symptom scores more than conventional treatment.

That more cautionary detail should have made its way into the story. We did like that the story mentions this is by no means a cure, and that children will still need plenty of support growing up. But saying that researchers have found a “potential breakthrough” for results that might have been due to chance seems misleading.


Why This Matters

It’s hard to overstate the desperation of families with severely autistic children who often cannot speak, follow any instructions, or communicate effectively in substantive, predictable, or manageable ways. To its great credit, the Guardian article does make clear that such families are often without hope and too often fall victim to unproven and false theories of origin and treatment, including the discredited claim that vaccines are responsible for the disorder. Any story about research suggesting even modest improvements in managing the core symptoms and behaviors of autism is likely, therefore, to garner wide attention. And although this study offers some promise that parental training and early intervention can help, care must be taken to reveal all of the findings and keep hyperbole out. A posting by Britain’s National Health Service also did a nice job of making this point. 


Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The article does a good job of noting the social, health, lost productivity and educational costs of autism in the UK and the United States, but it did not note the cost of the interventional program itself, and the time challenges that caregivers of autistic children face in pursuing this kind of intervention.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story attempts to quantify the benefits with this statement:

At the start of the trial, 50% of those in the control group who did not get the therapy and 55% of those who did were assessed as severely autistic. The children in the intervention group, though, got better. The proportion assessed as severe in the control group was 63% by the end of six years, compared with 46% in the intervention group.

But as we point out in the summary, there were some important limitations to this finding.

Does the story adequately explain/quantify the harms of the intervention?


The article quotes an outside expert who notes that the training intervention was non-invasive and non-intensive.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

Here the story missed important opportunities to put the findings in better perspective by noting the limitations of the study, including reliance on video excerpts (rather than direct observations) to assess parent-child interactions, and the variables in assessor training and definitions of severe autism.

Does the story commit disease-mongering?


There was no disease mongering, although the article should have made clearer that the study was conducted on children with a severe form of autism and did not involve children elsewhere on the “spectrum.”

Does the story use independent sources and identify conflicts of interest?


Outside sources were included in evaluating the impact of the study, including one from an autism advocacy group. But none of the experts noted limitations or cautions about over-interpreting the data in the long-term follow up. Almost every source was cheerleading.

Does the story compare the new approach with existing alternatives?


The article noted that there are no cures or predictably effective treatments for severe autism, and that even a modest improvement in symptoms would be a boon. The training intervention is compared adequately to the traditional “direct therapist-child” type of treatment.

Does the story establish the availability of the treatment/test/product/procedure?

Not Satisfactory

The article leads the reader to conclude that the training program is a “potential breakthrough” but says nothing about a) how many children would qualify for this training program in the UK; b) whether there are enough trainers; or c) how or when people inside or outside the National Health Service can find the program and sign up.

Does the story establish the true novelty of the approach?

Not Satisfactory

The story attempts to establish novelty with this statement:

“A new form of therapy has for the first time been shown to improve the symptoms and behaviour of autistic children, offering a potential breakthrough in care for millions of families.”

However, that’s erroneous: The investigators make clear in the study that there was a previous trial of PACT with promising results. What is clearly novel in this study was its size (152 children), randomization and long-term follow up.

Does the story appear to rely solely or largely on a news release?


The article appears to have involved interviews and original reporting of the research summary published in the Lancet.

Total Score: 5 of 10 Satisfactory


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.