Early report card on quality of FDA news releases: C at best

Posted By


Joy Victory is deputy managing editor of HealthNewsReview.org. She tweets as @thejoyvictory.

Last year for the first time, HealthNewsReview.org began reviewing health care PR news releases from a variety of sources using our 10-criteria system. We are not aware of another such effort to regularly and systematically analyze and grade such PR news releases.

As we publish this piece, we have reviewed 265 news releases so far. Five of those have been from the FDA–and we have some good news and bad news about the quality of their news releases.

The good news: For the most part, the releases have scored at least 3 out of 5 stars on our scoring system, which usually means getting satisfactory scores on 5 or 6 of our 10 criteria.

FDA news releasesThe bad news: None of the FDA’s news releases have satisfactorily met both of our two criteria that deal directly with medical evidence. The first criteria–“Did the PR news release adequately quantify the benefits of the intervention?”–looks at whether a news release adequately gives a sense of the scope of potential benefit.  In other words, how many people were helped out of how many who got the intervention and to what degree? The second–“Did the PR news release adequately convey the quality of the evidence?”–analyzes whether the release provides enough details on the quality of the evidence that was used to determine those benefits, as in, Was it a large, long-term randomized controlled trial or a small, short-term observational study?

These two criteria, when both reported adequately, provide the reader with the basic tools needed to assess the claims being made. One would expect that the FDA–which reviews reams of research before a product is approved–easily has that information on hand and could include it in news releases.

Instead, the news releases we’ve reviewed include mostly vague statements that may come across as endorsing new products without providing the context that readers need to make decisions. Admittedly, this is a small sample of the agency’s PR news releases and doesn’t support sweeping conclusions. But it does raise questions about the FDA’s diligence in providing balanced, thorough information to the public about the drugs it approves.

“They could be and should be one of our premier public health agencies, and could be taking the lead on explaining evidence that they evaluate and the benefits and risks of the drugs and devices they approve, along with uncertainties that always remain,” said Susan Molchan, MD, a psychiatrist in the Washington, DC area, and a HealthNewsReview.org contributor who also has worked as a medical officer for the FDA

What’s missing from FDA news releases?

So what would we like to see the agency do better on these criteria? Let’s take a look:

In order to give readers a sense of the size of the potential benefit from an intervention, a news release should provide the numbers behind the claim, aka the stuff found in the “results” section of a research paper. And readers need some context to understand the numbers, too.

Speaking hypothetically here, a release shouldn’t simply state that, say, a new drug cut heart attack risk in half. Instead, state that of 100 people divided equally into a control and intervention group, 2 had heart attacks in the control group, and 1 had a heart attack in the intervention group (a 50% relative risk reduction that sounds very impressive when framed that way, but a 1% absolute risk reduction for the people in the study). Our primer on absolute vs. relative risk has more detail on why this is important.

Sometimes the FDA does a pretty good job on this. In a news release on preliminary findings on a new bladder cancer drug, the agency did provide numbers on how many patients saw tumor shrinkage and for how long, but not the important context to understand what those numbers really translate to:

“Tumor shrinkage doesn’t always lead to improvements in survival or quality of life. It is a surrogate endpoint — a substitute — for the main outcome which is survival. Although this drug looks promising, to be fully transparent about its benefits as currently known, the release should have explained this,” our reviewers said in our review of the release.

But sometimes, the FDA doesn’t even include numbers, as we saw in our most recent review of a news release announcing the approval of expanded indications for a monoclonal antibody drug known as Ilaris.

FDA news releasesThe only nod to the evidence was this scanty detail: “Approvals for the new indications were based on clinical studies, including safety, efficacy and pharmacokinetic data.”

“Some variability across news releases is to be expected, as different staff may be responsible for writing different news releases,” said Matt Shipman, a public information officer at North Carolina State University and a contributor to HealthNewsReview.org. “However, every news release should provide fundamental information about the product that has been approved.”

He added, “benefits need to be quantified, not discussed in qualitative terms that are open to interpretation.”

And in a news release on a new drug approved to treat hallucinations and delusions associated with Parkinson’s disease, the FDA skips not just the quantified benefits, but also important limitations of the research, which included a small sample size and a short trial duration.

As our reviewers noted in our review, “We’re only told that, compared to a placebo, Nuplazid decreases the frequency and/or severity of hallucinations and delusions. But we aren’t given any actual numbers on how significant this improvement is.”

And this is important because “the FDA has approved many drugs that underwent far more extensive clinical trials that were later pulled from the market due to adverse events.”

Why this matters: The big picture

The lack of information is concerning because it overlaps with the a wider effort by federal entities to stonewall journalists, stifling transparency and even basic news gathering, notes Kathryn Foxhall, a freelance reporter focusing on Washington and member of the Society for Professional Journalists’ Freedom of Information committee.

Journalists, for example, must fill out an application with the FDA’s public information office anytime they want to speak to a staff person. This censorship creates a choke point few reporters can get through, she said.

“With such control an agency can tell the press and the public, ‘take it or leave it.’ We have little recourse,” Foxhall said. “Unfortunately, reporters also frequently have to go through strict controls to get information from companies that sponsor the products. So the public is being ridden around on a carousel of information control.”

Jim Dickinson, who runs the web site Dickinson’s FDA Webview, said publication information officers at the FDA are now effectively sealed off from journalists, and this affects how the agency communicates with the press.

“Their job is to help management control news coverage and keep journalists away from subject matter experts,” he said. “This has led to an often adversarial relationship between the PIOs and the reporters.”

“It has also led to less-informative news releases,” he added.

The approval of a new product typically generates an avalanche of news coverage, and the FDA is in a unique–if not vital–position to improve that coverage by providing the information that journalists and the public need. The FDA’s stamp of approval suggests to the public that a new intervention works and is safe.

But how well does it work and what are the potential harms and the limitations of the evidence? News releases from the FDA could go a long way toward helping journalists – and the public – improve their own critical thinking about drugs and devices. Or, it can be lost opportunity.

You might also like


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Comments are closed.