Is 1 in 7 Odds a Breakthrough? Is it Even Promising?

Hand Rolling Dices Isolated on White BackgroundThe following is a guest post from Andrew Holtz, one of our longtime contributors, who is an independent health journalist based in Portland, Oregon. Andrew has been a medical correspondent for CNN and is past president of the Association of Health Care Journalists.  

One in seven. That’s how many cancer drugs approved using surrogate endpoints (such as tumor response or progression-free survival) actually demonstrated they helped patients live longer, according to a study that looked at the evidence available on such drugs more than four years after their approval.

That track record is important because news reports… and patient hopes… shoot up, with glowing talk of breakthroughs or at least promising results, based on exactly this sort of surrogate endpoint study. And why shouldn’t they? After all, when the US Food and Drug Administration approves a drug, that means it works, right? Well, that depends on your definition of “works”.

Here’s what the study authors, Chul Kim, MD, MPH and Vinay Prasad, MD, MPH, reported in JAMA Internal Medicine. They found that from 2008 through 2012, the FDA approved 36 cancer drugs based on surrogate endpoints; that is, the FDA didn’t require drug makers to wait until they finished studies demonstrating patients lived longer. Drugs can be approved based on such evidence when they are given “breakthrough” designation by the FDA, which means that the drug “treats a serious or life-threatening condition” and “may demonstrate a substantial improvement…over available therapies.” There is understandable demand to get new drugs for serious conditions into the clinic. But when a drug is approved based on its ability to shrink tumors or at least stall their progression, drug makers are supposed to keep studying… in order to find out whether or not patients actually live longer, which is, after all, the main point of treatment.

The problem is that after all the initial hope and hype… only five of these 36 drugs were shown to actually help people live longer (see figure). Eighteen of the drugs failed to show real survival benefits in follow-up studies. And for the remaining 13 there was just silence. After more than four years either they had not been tested or survival results had not been reported.

cancer drug figure

Credit: Kim and Prasad. JAMA Intern Med. Published online October 19, 2015.

The researchers point to Avastin (bevacizumab) for metastatic breast cancer as the poster child, not in a good way, of this post-approval-party hangover. The drug was approved for this use based on study results showing better progression-free survival. But the hoped-for real survival boost not only failed to materialize for these patients, experience revealed harm from side effects serious enough that the FDA ordered Avastin’s maker to add warnings to the drug label for hazards including “ovarian failure in premenopausal women, osteonecrosis of the jaw, and the risks of venous thromboembolic events”.

So rather than headlining the health report with “The FDA has approved a promising new cancer drug,” perhaps it would be more accurate, if less exciting, to report, “The FDA has approved a new cancer drug that based on recent experience probably won’t work.” Okay, maybe that line is too much of a downer. But journalists can at least be more diligent about applying good old skepticism to reports that a new drug has shown it can shrink or stall tumors, because based on this review of recent experience, odds are those surrogate endpoints have only a one in seven chance of translating into something people really care about: longer life.

Publisher’s note from Gary Schwitzer:  on Twitter, Milwaukee Journal-Sentinel reporter John Fauber wrote, “Gratifying to learn that a JAMA Internal Medicine paper on new cancer drug approvals confirms our investigation from last October.” It should be gratifying.  Fauber’s piece was important journalism.  If you haven’t read it, I encourage you to do so. 

You might also like

Comments (3)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Paul Raeburn

October 26, 2015 at 9:52 am

Interesting piece, and it’s always a good idea to remind us that progress against surrogate endpoints often does not lead to progress against cancer. But what about the patients whose lives were extended by the five drugs that were later shown to be effective? If these 36 drugs had not been approved based on surrogate endpoints, those patients might well have died. Approving drugs based on surrogate endpoints means ineffective drugs will get through the system. But it also means that lives will be saved.

    Andrew Holtz

    October 26, 2015 at 6:31 pm

    Indeed, Paul is quite right that five of the 36 drugs were able to show they extended the average survival of cancer patients, thus fulfilling the purpose of the accelerated approval process, including reliance on some surrogate endpoints. My quibble is not with the FDA process, but with the often baseless assumption of news headlines that these new drugs are probably going to extend lives. They might. When they do, that’s wonderful. But this study is another warning (too often unheeded) that surrogate endpoints are not the same as extended survival, or even meaningful quality of life improvements.
    I don’t think I need to do a thorough review of news stories about these cancer drugs to predict that while many of them grabbed attention when first approved, there was little reporting of follow-up studies when they showed no survival benefit. We have a responsibility to make crystal clear that approvals based on surrogate endpoints, while they may be the right thing to do, represent Hail Mary passes that more often than not fall incomplete.

Marc Girard

October 27, 2015 at 6:43 am

Paul’s comment overlooks important issues: i) the reliability of the clinical trials supposed to confirm a real benefit for 5 drugs, which is far from being certain (having regards to biases such as the funding effect); ii) the classical dearth of reporting as far as adverse effects are concerned (esp. in clinical trials on cancer, as many of the drugs hazards are ascribed to the natural evolution of the underlying disease), resulting in fantastic distorsions of the real benefit/risk ratio of the drugs in question; iii) the grey area related to the “silence” on 13 “promising” drugs, which is likely to hide unexpected (and maybe frightening) results on the long run; iv) the morbi-mortality related to the unavailability of essential drugs, ascribable to inadequate allocation of ressources; v) the dramatic health consequences of the economical crisis caused by the indecent behaviour of the main capitalist agents, as perfectly illustrated by the “organised crime” pointed out by Gotzsche and perfectly exemplified by the contrast between the cost of new anticancer drugs as compared to their genuine achievements…