Poor to fair improvement in health of HIV patients described as “amazing”

Does the news release adequately discuss the costs of the intervention?

Not Satisfactory

We did not find any discussion of the cost of bringing patients in for five sessions of what is called “positive skills intervention.” While it may be low-cost, we need some mention of that to help evaluate the practicality of this therapy.

The study itself devotes space to cost analysis.

“The [intervention] was quite time intensive for both the study team and the participants which lessens the likelihood that it can go to scale in this format.”

“We have begun work on translating the intervention to a self-guided online format that would significantly reduce the cost.”

If the news release had included these details, it would have added some practical context for readers.

Does the news release adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

We were confused by the statement below and the contrasting information in the journal manuscript reporting the study results. First, let’s see what the release claimed as benefit:

“Fifteen months after the interventions, 91 percent of the intervention group had a suppressed viral load compared to 76 percent of the control group. In addition to the potential benefit of a lower viral load on the infected person, there may be public health benefits.”

And the quoted researcher called this an “amazing” difference.

But this is what the study abstract stated:

“This comparatively brief positive affects skills intervention achieved modest improvements in psychological health, and may have the potential to support adjustment to a new HIV diagnosis.”

The results of the study itself are very mixed; it describes some of the measurements of the study as “not reaching statistical significance.” It seems clear that the intervention group did have improved (or at least not decreased) psychological health, which is a reasonable outcome for having gotten psychological attention and therapy.

Does the news release adequately explain/quantify the harms of the intervention?

Not Satisfactory

If the researchers believe the intervention is without harm, the release needed to spell that out. Harms were also not measured or disclosed in the study itself, but it’s something we encourage news release writers and journalists to seek to clarify.

Does the news release seem to grasp the quality of the evidence?

Not Satisfactory

There are several issues related to the quality of the evidence that we think this news release left out.

Since the differences found in the study between the trained group and the controls were relatively small (the effect sizes under 0.4, which is poor to fair only), it is likely that modifying the intervention without the face-to-face intervention with an engaged volunteer would result in lower differences and may actually be identical to the control group.

A couple of other things were worth mentioning in the release related to quality of evidence. It was a randomized study but there is no mention in the release about blinding of patients or observers, the absence of which could lead to bias in the results of the psychological observations. There were also a large number of patients who did not qualify for the study, making generalization of the study a problem.

Finally, it would help readers to know that studies have raised questions of the reproducibility of much psychological research.  In other words, when the same experiment is tried by others, it often doesn’t turn out the same way.  It might be too much to expect a news release to delve into this topic, but we include it herein for the benefit of our readers.

Does the news release compare the new approach with existing alternatives?

Not Satisfactory

The release does not mention support groups, meditation or other behavioral management training programs that exist to help patients cope with chronic illnesses, nor does it say how this intervention might compare with other therapies currently used in clinical practice.

Does the news release establish the true novelty of the approach?

Not Satisfactory

The release contains this claim of novelty:

“This is believed to be the first test of a positive emotion intervention in people newly diagnosed with HIV. Based on the study results, the intervention is promising for people in the initial stages of adjustment to any serious chronic illness.”

We believe others in medicine have probably used “positive” intervention before, but they may have called it support or emotional coaching or many other phrases. Cognitive-behavioral therapy, massage, and other means to improve a person’s coping skills could be considered “positive emotion intervention.”

Does the news release include unjustifiable, sensational language, including in the quotes of researchers?

Not Satisfactory

We find the word “amazing” used to describe the results as too much of an exaggeration for a study that in its peer reviewed journal format is called “modest” for results. The best that the study showed was a fair to poor difference between the two groups of patients studied.