Read Original Release

A “sniff test” for autism passes the smell test — with reservations

Rating

2 Star

Tags

Can autism be measured in a sniff?

Our Review Summary

Boy with nose held shutThis news release describes a recent paper in the journal Current Biology which reports that children diagnosed with autism spectrum disorder (ASD) do not respond to unpleasant smells in the same way that other children do. Specifically, children with ASD were found not to vary their sniffing behavior regardless of whether they were presented with pleasant or unpleasant smells; whereas children without ASD would sniff less when presented with an unpleasant smell, for example. The release says that this test based on a smell — a nonverbal stimulus — could be used in very young children to diagnose the condition early, when it’s important to begin treatment. The release does a solid job of explaining this intriguing research and laying out the implications in broad terms — particularly that much additional research needs to be done before such a “sniff test” could be used in a clinical setting. The inclusion of a few more contextual details, as discussed below in the review, would have increased the value for readers as well as our star score rating for the release.

 

Why This Matters

According to the CDC, in 2010 one out of every 68 children is diagnosed with ASD — so this is a subject that’s relevant to a lot families. And the CDC also says that “research shows that early intervention treatment services can greatly improve a child’s development.” But diagnosing ASD can be difficult. Given that ASD is relatively common, and that an early diagnosis may make a significant difference for children with ASD, any development that would facilitate an early diagnosis is probably worth writing about. This news release covers early stage research, but the research is interesting — and the release makes clear that significant additional work needs to be done before it could be incorporated into widespread diagnostic testing.

Criteria

Does the news release adequately discuss the costs of the intervention?

Not Satisfactory

The release doesn’t address cost at all. And while the release notes that a so-called “sniff test” is still far from ready for clinical use, it does expressly discuss the possibility of using such a test as a diagnostic tool. We feel that if it is worth discussing a finding’s clinical application, it’s worth discussing the potential cost. We understand that it may be difficult or impossible to place a dollar value on such a diagnostic tool, but the release likely could have discussed the cost in general terms. For example, does it rely on expensive technologies or specialist skills? Or is it something that could be done by a trained general practitioner with inexpensive items? Or do we simply not know yet? Regardless, a quick sentence or two could have shed some light on the matter.

Does the news release adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The release mentions a diagnostic accuracy rate of 81%, but where exactly does this number come from and what does the term “accuracy” mean here? We’d prefer a more detailed discussion that includes an explanation of the sensitivity and specificity of the test. Or at least a mention of false positive and false negatives. The study that’s the basis for the test told us there was 1 false positive and 6 false negatives out of the 18 people in each group. But later on in the release, we learn that the researchers aren’t sure yet if the test can distinguish between autism and other similar neurodevelopmental conditions, so the low number of false-positives reported may not hold up with additional research. In addition, the release says that while typical children adjusted their sniffing almost instantly when smelling an odor, “children on the autism spectrum showed no such response.” If that’s the case, why wasn’t the test 100% accurate?

The release did a reasonable job here and emphasized that the study isn’t ready for clinical use. But we don’t think it quite met our standard for reporting on the complex issue of diagnostic accuracy.

Does the news release adequately explain/quantify the harms of the intervention?

Not Satisfactory

The problem with any diagnostic tool, including very early-stage diagnostic concepts like this one, is that while they can offer accurate diagnoses — they may also provide an inaccurate diagnosis. This is particularly relevant, since the release quotes one researcher as saying the work “raises the hope that these findings could form the base for development of a diagnostic tool that can be applied very early on, such as in toddlers only a few months old.” But here’s the thing: in the event of a “false positive” diagnosis of ASD in a very young child, it may take months or years for the diagnosis to be disproven. That could cause unnecessary stress for the patient and family and — given the expense of ASD treatment for many families — could impose a significant economic burden as well. The damage would be minimized by using the test, as any test should be, in the context of a good clinical evaluation.

Does the news release seem to grasp the quality of the evidence?

Satisfactory

The release clearly describes the study, including the study size, the use of a control group, and the composition of the study participants. The release could have been even stronger, however, if it had placed that information in context for readers. For example, the study evaluated 18 children diagnosed with ASD and 18 children without ASD. Is that a robust sample size for a study of this kind? Most readers likely won’t know. In addition, the symptoms of autism can range from mild to severely debilitating. Where were these children on that spectrum, and what does that say about the test’s usefulness for differentiating between normal children and those with mild impairment? The release tips toward a Satisfactory rating with its emphasis on the fact the test is not ready to be used clinically.  “The findings suggest that a sniff test could be quite useful in the clinic, although the researchers emphasize that their test is in no way ready for that yet.” Well said.

Does the news release commit disease-mongering?

Satisfactory

No disease mongering here.

Does the news release identify funding sources & disclose conflicts of interest?

Not Satisfactory

The release does not tell readers who funded the research. The release also doesn’t mention the fact that, as the paper itself states, “The results of this manuscript are under consideration of patent at the Weizmann Institute Office of Technology Transfer.” That’s worth mentioning, particularly since the release quotes the study’s corresponding author, who is on faculty at the Weizmann Institute of Science. This isn’t an unusual state of affairs for potentially useful research being done at large institutions, but we feel it is something that a release should share with readers.

Does the news release compare the new approach with existing alternatives?

Not Satisfactory

The release infers that a new tool for early diagnosis of ASD in children would be valuable, but it doesn’t tell readers why. Readers would be better able to appreciate the significance of the new work if it had addressed — even in broad terms — the challenges involved in current approaches to diagnosing ASD. What do ASD screening and diagnostic techniques look like for infants? Toddlers? Older children? That information would be valuable context for understanding the potential importance of the new study.

Does the news release establish the availability of the treatment/test/product/procedure?

Satisfactory

The release not only makes clear that a sniff test is “in no way ready” for clinical use, but lays out several questions for relevant future research that needs to be done before such a clinical tool can be developed.

Does the news release establish the true novelty of the approach?

Not Satisfactory

The release does mention that people with ASD have impairments in their “internal action models,” and defines these models as “templates we rely on to coordinate our senses and actions.” But, the release notes that it wasn’t previously clear “whether this impairment would show up in a test of the sniff response.” Fair enough. But researchers have published a number of papers over the years addressing various sensory differences between children with ASD and children who are developing normally (like this one, and this one, and this one). In other words, while the Current Biology paper discussed in this release is clearly new, it fits into a continuum of research on ASD and sensory sensitivity (including smell). Discussing how the new work fits into that continuum, even in a sentence or two, would have given readers valuable context.

Does the news release include unjustifiable, sensational language, including in the quotes of researchers?

Satisfactory

The release did not over-sell the findings, and was very careful to point out the work that needs to be done before a clinical test can be put into widespread use.

Total Score: 4 of 10 Satisfactory

Comments (1)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Jon Brock

July 7, 2015 at 1:10 pm

When talking about potential diagnostic tests it’s also important to consider base rates – the prevalence of the condition in the population being tested. The authors report that 12/18 autistic kids and 1/18 typically developing kids tested positive. But in real life there are 67 typically developing kids for every 1 autistic kid (according to the CDC figures). So you’d actually expect 67 false positives for every 12 true positives. In other words only 15% of positives are true positives.

Reply