This release on a suite of smartphone apps designed to reduce depression and anxiety comes from a prestigious university and was funded by the National Institutes of Health. Issued under embargo, an emailed version of the release pitched journalists with the titillating subject heading: “Depression Apps as Fast as Finding Good Sushi.”
But if Northwestern University wants us to think of this app as the Yelp of depression treatment, it will need to serve up more convincing evidence than what was provided in this skimpy news release. After all, treating depression is a little more complicated than locating a good California roll, and we expect to see substantial backing for the suggestion that these apps work as well as clinician-delivered psychotherapy or antidepressant medication. The release’s comparison of the app to medication is fatally flawed because more than half of the participants were already taking antidepressants when they enrolled in the study, and about 20% were receiving psychotherapy. How much of the benefit was due to the app and how much to these or other factors is impossible to say.
Bottom line: while the release deserves some credit for highlighting one major weakness of the study — the lack of a control group — it didn’t mention other key limitations or adequately describe the scope of the benefits seen. Like the sushi lovers courted in the subject heading, anyone feasting on this release is likely to be hungry again in an hour — if not sooner.
People are being pitched on new health-related smartphone apps on a weekly basis. By now, almost everyone is aware that that they can access health interventions via the Internet and one would hope these apps are becoming more evidence-driven. Efforts such as Northwestern’s to give more people access to coping tools while assessing their effectiveness is laudable and forward-looking.
It’s even more impressive when considering that there’s scant evidence to back up claims made by scores of developers of apps for people who need help dealing with anxiety and depression. According to an April 2016 feature article in Nature, “But the bare fact is that most apps haven’t been tested at all. A 2013 review identified more than 1,500 depression-related apps in commercial app stores but just 32 published research papers on the subject. In another study published that year, Australian researchers applied even more stringent criteria, searching the scientific literature for papers that assessed how commercially available apps affected mental-health symptoms or disorders. They found eight papers on five different apps.”
The release notes that there is no cost associated with downloading the Intellicare apps. But because Intellicare is part of “a national research study” the researchers request that users provide “confidential feedback” weekly to help them further develop the system. Some people may not be comfortable sharing personal data over the internet, with good reason.
The release makes several claims but doesn’t back them up with data. For example:
“13 novel mini-apps reduced depression and anxiety by nearly 50 percent in a preliminary research study”
What kind of depression and anxiety are we talking about here? Feeling a little blue or so severely depressed you can’t get out of bed? Which tools were used to measure or rate participants’ level of depression or anxiety?
“participants reporting significantly less depression and anxiety by using the apps on their smartphones up to four times a day”
Again, what does “significantly” refer to?
“The short-term study-related reductions are comparable to results expected in clinical practice using psychotherapy or with that seen using antidepressant medication.”
Claiming that using the apps for 8 weeks is comparable to a similar period of psychotherapy or antidepressant use is unfounded without a randomized clinical trial. The study itself notes that 64% of participants were taking antidepressant medication, and 22% were receiving psychotherapy, when they enrolled in the study.
Although there’s little harm associated with using any app on the face of it, relying on an app instead of seeking professional help could be harmful. The release should have acknowledged this, and done a better job of explaining who is most likely to be helped by a smartphone app and which individuals have symptoms severe enough to warrant seeking professional medical assistance. There are many different types of depression that range in severity. The National Institute of Mental Health describes several — seasonal affective disorder, persistent depressive disorder, major depressive disorder, psychotic depression, postpartum depression.
The release notes that 96 of the 105 initial volunteers completed the study and that participants received coaching (including phone calls and texts) during the 8 week study. But it didn’t explain that maybe it was this human coaching, and not the apps per se, that was at least partly responsible for the benefits seen in the study. The researchers acknowledge in their manuscript that “we cannot disentangle the effects of the coaching and the apps.”
The release also could have done a better job of highlighting the limitations of such a short study. Will users get bored with the app and see their depression/anxiety return when they stop using it?
We’ll give credit to the release, however, for nicely describing a major weakness of the study. It states: “The preliminary study did not include a control arm, so it’s possible that some people who enrolled in the trial would have improved anyway, partly because they may have been motivated to try something new, Mohr said.”
The release doesn’t go overboard describing the prevalence of depression in our society. It reasonably states, “More than 20 percent of Americans have significant symptoms of depression or anxiety each year, but only around 20 percent of people with a mental health problem get adequate treatment.”
The release notes that the study was funded by the National Institute of Mental Health.
The release states that over the course of 8 weeks, participants experienced improvements similar to what people with depression or anxiety would find through psychotherapy or medication. But again,how can such a comparison be made when 64% of participants were already taking antidepressant medication, and 22% were receiving psychotherapy, when they enrolled in the study?
The release invites people to download the apps for free.
The releases says, “Most apps designed for mental health typically offer a single strategy to feel better or provide too many features that make them difficult to navigate. Users may get bored or overwhelmed and may stop using the apps after a few weeks.”
This statement doesn’t adequately explain why the same problems wouldn’t apply to the apps described in the release. (After all, the study was only 8 weeks long.) Nor are we told how these apps differ from the thousands of others on the market.
The release doesn’t rely on sensational language, although the sushi analogy was ill-advised.