Carolina Branson, PhD, is an associate editor with HealthNewsReview.org. Her graduate work at the University of Minnesota focused on media studies and health.
The sale of testosterone is a booming business. Some men take supplements of the hormone hoping to increase libido, energy, muscle mass, and more. But until now, there was little evidence that using testosterone would provide any of these hoped-for benefits. Enter a new study published in the New England Journal of Medicine that reports modest positive effects for men over 65 with documented low testosterone who spread a testosterone gel on their abdomens. The study was the first of its kind to establish that using a testosterone gel could deliver an effect more powerful than a placebo.
But the improvements experienced by these older men were hardly anything to write home about (see figure, left). The primary outcome was assessed via a questionnaire that measured participants’ sexual activity on a scale of 0 to 12. After a short-term boost of about a half a point, the difference between the testosterone group and placebo dipped to 0.3 points after a year — barely noticeable. And it’s likely that some of the 230 men receiving testosterone therapy were no better off than if they’d been using skin cream.
And yet the personal patient stories featured in news coverage painted a rosy portrait of the benefit. Consider this story from NPR that recounted an anecdote from Dave Bostick, a patient who…
…it turned out, was in the group assigned testosterone. He did develop a more robust libido, he says. But even more important, he adds, he felt happier. He had more energy, and his enthusiasm for trying new things — new classes, new interests — was restored. His part of the study ended two years ago, but Bostick is still taking testosterone.
The New York Times, interestingly, also turned to Bostick for insight into how testosterone treatment made him feel:
For David Bostick, a 71-year-old Pittsburgh man who participated in the study, the appeal of testosterone was that it might help with his sluggish feeling and reduced libido. After a few weeks of smearing a gel—he did not know if it was Androgel or a placebo—on his stomach, he began to feel different, with more sexual desire and more energy. He guessed he must be receiving testosterone.
NBC, meanwhile, turned to an orthopedic surgeon — Dr. William Reilly — who “helps run a chain of testosterone treatment facilities called the Low T Centers” for a first-hand account of the effects. With that background, is it any wonder that Reilly is enthused about the impact testosterone treatment had on him? Reilly tells NBC that he “hit a brick wall” when he was 62 and found that “all of a sudden my energy level just wasn’t the same.”
He started getting less out of his gym workouts. “My arms were starting to get smaller. My belly was getting bigger,” Reilly said.Reilly said testosterone replacement therapy helped him feel better.
Ed Stevens of Ft. Lauderdale is also quoted by NBC as saying, “I wouldn’t call it a huge difference. I just saw over the months my body mass came back to where it was, my energy was better, libido was a little better.”
Compared with some story anecdotes we’ve seen that can wildly mischaracterize benefits, these examples are not egregious. The stories we looked at all provide a fair accounting of the study findings and pay appropriate attention to potential harms. But the focus on single patient anecdotes — universally touting positive results — is troubling. As we note in the description of our review criteria:
Patient vignettes make engaging reading but they are not data. When a story is top heavy with personal stories, it makes it hard for readers to sustain their critical thinking when (if) they get to information that is quantitative. If you hear glowing patient anecdotes about how well something worked, always ask yourself if that was a representative example.
If we’re going to play up anecdotes, why not feature the guy who achieved better results using Viagra?
Why not profile the guy who couldn’t tell if he was using testosterone therapy or skin cream?
Why not look at the guy whose PSA levels went up on testosterone and who was worried about his prostate cancer risk?
There may be a reason that the study itself, and thus the subsequent coverage, tended to focus more heavily on the positive outcomes: a pharmaceutical company partially funded the study. According to Australian journalist Ray Moynihan, who evaluated both the NBC and the New York Times stories, these outlets did not adequately address these conflicts of interest in the study. Moynihan has been a vocal critic of attempts to medicalize sexual problems by the pharmaceutical industry, and he thinks these stories could have done more to notify readers of the researchers’ links to drug companies.
In my view the NBC web-story came across as more promotional than journalistic, it doesn’t seem to have any independent critical comment in it, and the story makes no mention of a crucial fact – the study was co-funded by the gel maker.
The Times story does contain some critical questioning voices and covers the issue of potential side effects, and while it does mention the drug-maker co-funded the study, its fails to cover the fact that 6 of the study researchers had direct ties to the drug maker, and a total of 16 researchers disclosed ties to pharmaceutical or supplement makers – including being paid directly as advisors or consultants.
We know from much evidence that sponsored studies tend to find results more favourable to the sponsors products – and having so many authors with such close ties to so many drug -makers raises questions about the reliability of these findings and their interpretation. In my view, those questions were not given enough airplay in either of these stories.
One of the things the Times coverage does well is emphasize the lack of long-term data about potential heart and cancer risks associated with testosterone therapy. Concerned about the huge and expanding market for testosterone treatments, Dr. Doug Campos-Outcault — one of our regular story and news release reviewers — hopes this message is one that will resonate with Times readers.
“This is a very classic example of the ready, shoot, aim approach to medicine we take in the USA,” Campos-Outcalt said, “implementing widespread use of a therapy that has not been adequately tested for benefits and harms.”