This report from the New York Times is a win for almost everyone. The paper gets a big scoop and a drug company gets a nice dose of front-page publicity for its experimental drug. Who loses? Anyone who think it’s important to carefully assess what happens in medical research so that good decisions can be made about treatment (i.e. the rest of us).
We can understand why Novartis would choose to selectively disclose the results of an important study to a reporter and then have him collect comments about the "surprising results" from others who haven’t seen the data. Such a strategy seems likely to yield plenty of positive reactions, as this story predictably reported, while limiting opportunities for unwanted hole-poking. But we think it’s unfortunate that the Times chose to play along with Novartis’s game. In our view, an interview with a hand-picked reporter is not the appropriate venue to release the results of important new medical research, and it seems naive for the Times to pass along Novartis’s perspective on the findings without being allowed to corroborate it with other expert sources. While this drug could eventually live up to its billing as "perhaps the most promising therapeutic discovery ever for a gene-based behavioral disease," as one researcher breathlessly put it, we’d prefer to at least see a study abstract before seeing these kinds of claims put down in print for all the world to base their hopes on.
It’s unfortunate that these issues obscured some of the interesting background on fragile X and the research process, which was informative and engaging.
Mental retardation is a condition that places a huge burden on millions of patients and their families. Treatment options are limited and generally not very effective. A new drug that addresses the source of the problem would certainly be an important development worthy of front-page coverage in the New York Times. However, it is critical not to instill false hope by overhyping experimental drugs before their benefits are proven.
Since the drug is in such an early stage of development, we’ll call this one not applicable.
Again, the story falls down here when it passes along a Novartis researcher’s vague account of what happened in the study instead of providing us with actual statistics. The researcher says there was "clear improvement" in the treated patients, but the story should have pressed for a more detailed account of how many patients benefited, by how much, and according to what outcome.
Also unsatisfactory was the rampant speculation about who else might benefit from the drug beyond patients with fragile X syndrome. According to the Novartis account of the research, only a subgroup of patients with an "undisclosed biological trait" benefited from therapy with the experimental drug. And this was in a study that was already extremely small ("a few dozen patients") to begin with. But The Times quotes three different sources who suggest, based on no evaluation of the data, that the drug might be useful for a broader spectrum of autism disorders. And the quotes include some over-the-top language about the drug perhaps being "the key to solving the mystery of autism and other developmental disorders."
By relentlessly playing up the positives and failing to rein in these more exuberant proponents, we think the story gives an unbalanced view of progress in the development of this drug and what it will likely be capable of. The fact is that very few drugs in this stage of development will be approved by the FDA and become available to patients.
One of the big problems of reporting on research that hasn’t been published or peer-reviewed is that harms don’t get adequately quantified–a drawback that was on full display in this story. The report didn’t even mention the possibility of patients being harmed by this experimental drug.
The story does an excellent job of explaining the science behind fragile X syndrome and the discovery process that led to the new investigational drug. And at first glance, the story seems to call attention to all the appropriate caveats and limitations that apply to early-stage drug research. We are told that the drug is years away from approval and that it might fail in future clinical trials. The Times also warns us that the trial results were not published or peer reviewed.
Troubling, however, is the fact that other researchers who commented for the story were doing so without having reviewed the Novartis data and had to be told by the reporter what the study found. That doesn’t strike us as a recipe for collecting useful feedback about complex medical research. Also problematic is the story’s characterization of the study outcomes reported by the researchers. We are told that an "undisclosed biological trait" improved in the treated group more than in the control group. The story says that these and other details about the study could not be disclosed by Novartis for commercial reasons. So the question inevitably arises: If the limitations and restrictions on a study are such that a reporter can’t get independent feedback about it and can’t even tell us what benefits were found, is the newspaper really justified in spending 1800 words covering the story? If the only evidence you are allowed to report on is a company researcher’s assurance that "clear improvements in behavior" were found, we think it’s a good sign the story might need a little more seasoning before it gets splashed across the front page.
This story does not resort to disease-mongering.
While the story does collect feedback from a number of independent expert sources, we think the way these sources were used was inappropriate and potentially misleading. The reporter seems to have approached many of the sources and asked them to comment– without the benefit of any actual data–on the reporter’s second-hand description of the apparent positive results of Novartis’s research. The sources all respond with a variation on, "Wow, that sounds wonderful." And really, who wouldn’t say the same thing? Nobody doubts that a drug effective for fragile X syndrome would represent a huge medical breakthrough. But the more important question prompted by the article is whether such a breakthrough has in fact occurred. Although there are some cautions and caveats sprinkled throughout the article, we think the extremely positive tone of the expert quotes far outweighs these nuggets of restraint. What bears repeating (and what we think gets lost in this story) is that these experts are enthusiastic about the concept of an effective treatment; they are not endorsing Novartis’s claim that the drug caused "clear improvements" in behavior.
The treatment approach appears to be genuinely novel. An expert source comments that fragile X syndrome, up until recently, was viewed as "disability needing rehab, not a disorder needing medication."
There’s no question that the drug discussed in the story is years away from being available and may never get approved by the FDA.
This drug does by all accounts represents a novel approach to the treatment of fragile X syndrome.
It’s clear that this story wasn’t based on a press release. But do we need to create a cateogry for stories based largely on non-published disclosures in conversations with pharma execs?
Systematic, Criteria-Driven
Monday at #ADA2021, an infectious disease epidemiologist, a journalist, and a researcher shared perspectives on communication of risk in the era of COVID-19: http://ow.ly/ROfR30rMeE7 @AstleyCM @garyschwitzer @berthahidalgo @ADA_DiabetesPro
Shoutouts to @susan_bewley @drkevinknopf @MichaelBaum11 in this call for more than fawning coverage from press releases for these kinds of stories.
Also, linked history inside goes back 5 years just on Grail apparently trying to become Holy Grail. https://twitter.com/garyschwitzer/status/1408849696416841731
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like