“New drug improves bone density by 40%.”
At first glance, this sounds like great news. But there’s a problem: We have no idea if this means the drug also cuts the risk of bone fractures, which is the outcome that we really care about.
So why do researchers measure bone density instead of fractures? For several reasons, it can be difficult to determine whether a treatment will result in a clear benefit for patients, such as preventing death or improving quality of life. It may take decades, for example, to see if a new osteoporosis drug ultimately reduces fractures, so researchers look for what are hopefully reliable indirect markers to measure, such as bone density.
These substitutes, which go by several names–surrogate measures, markers or endpoints–ideally can be assessed quickly and easily and are expected to correlate with a meaningful outcome.
Surrogates can be used to end a trial if they move in the wrong direction. They can also speed the development of life-saving treatments, which is the primary reason the FDA often accepts surrogates as evidence of a drug’s efficacy in clinical trials.
Since the early 1990s–when surrogate markers such as T cell counts and viral load were used to hasten the availability of HIV drugs–the FDA has allowed accelerated approval of drugs for serious diseases based on surrogates deemed “reasonably likely” to predict a real outcome. In this example, reduced viral load may signal a drug is working, but if it doesn’t lead to something meaningful, like preventing death, is it really effective? (In the case of HIV, viral load has been established as a useful way to tell if a drug works.)
A dangerous downside
A quarter century later, a major drawback has become clear: Not all surrogate measures have turned out to be good ones. Often a drug that influences a surrogate measure turns out to produce no meaningful result for patients, referred to as a clinical outcome.
In some cases, there is even harm. In the landmark Cardiac Arrhythmia Suppression Trial (CAST), drugs approved for their ability to suppress a surrogate marker — abnormal heartbeats — were found to actually increase rather than decrease the risk of death.
Similar results have been observed with diabetes medications that lower hemoglobin A1c (HbA1c), a measure of average blood sugar levels over the preceding three months. Although many such medications were approved based on their ability to lower HbA1c, a surrogate, research increasingly shows that these medications have very different effects on the risk of cardiovascular disease. Some medications lower risk while others may increase it.
Some say drugmakers have come to rely too heavily on surrogates for a wide range of conditions. But surrogate markers are common: Between 2010 and 2012, the FDA says it approved 45 percent of new drugs on the basis of a surrogate endpoint.
Surrogate measures have become the norm in several diseases. One analysis showed that 67 percent of cancer drug approvals over a five-year period were based on surrogates. From 2003 to 2012 the FDA used surrogates for seven out of nine drugs approved for chronic obstructive pulmonary disease, all 26 approved drugs for diabetes, and all nine drugs approved for glaucoma; in only a few cases did surrogate measures predict actual benefits for patients.
Drugmakers are supposed to submit follow-up studies after approval to determine whether a drug produces a clinical benefit. Research has shown that those studies often are delayed for years and when they do get done, real benefits can be elusive; even five years after approval, only a tiny percentage of drugs tested with surrogates were proven to be effective in rigorous trials.
Growing reliance on surrogate measures
The use of weak surrogate-based evidence has flooded the market with expensive duds, many argue. That’s especially concerning in cancer: Drugs are frequently approved on the basis of uncertain markers such as “progression free survival,” which is the amount of time between treatment and worsening of symptoms. The drug Avastin won accelerated FDA approval to treat metastatic breast cancer based on its ability to delay tumor growth, but that approval was revoked when multiple randomized trials showed the drug didn’t improve survival and had significant side effects.
As HealthNewsReview.org blogger Susan Molchan, M.D., noted: “This is a boon for drug companies, but not for patients, and some scientists think it discourages innovation, as companies tend to continue to take this easy route to approval, a sort of ‘teaching to the test’ of FDA approval rather than towards clinically meaningful benefits.”
It’s been proposed that drugs approved with “uncertain evidence of clinical benefit” be required to offer price concessions until they are confirmed to work. But don’t hold your breath on that.
Examples of good news coverage
It’s important for news stories to draw attention to the use of surrogate markers and explain how they limit the reliability of study results in predicting a benefit that matters to patients. The Slippery Slope: Is a Surrogate Endpoint Evidence of EfficacyHere are some examples of stories that did that:
- On research linking moderate alcohol consumption and bone health, TIME.com wrote that researchers “tracked certain blood markers of bone health throughout, and found that these markers of bone density correlated positively with alcohol consumption.” As our publisher Gary Schwitzer noted, the story also provided this important context: “Although there is substantial evidence that moderate alcohol consumption correlates with higher bone mass density in postmenopausal women, it is much less clear whether consuming alcohol lowers the fracture rate,” the authors note.”
- When two drug companies reported they were pulling the plug on Alzheimer’s drug bapineuzumab after two failed clinical trials, Bloomberg News reported: “While companies have focused on developing drugs to hinder the amyloid deposits, scientists aren’t certain whether the clumps cause or are a minor contributor to the disease or merely a consequence.”
- A New York Times story about the FDA’s accelerated approval of an immunotherapy drug for bladder cancer reported that the drug, Tecentriq, shrank tumors significantly in 14.8 percent of the patients, and the cancer completely disappeared in 5.5 percent, adding: “It is not yet clear whether the drug will allow people to live longer.”
- John Fauber and Elbert Chu, writing for MedPage Today and the Milwaukee Journal Sentinel, examined the many powerful forces driving greater reliance on surrogates in drug development — and how those influences may ultimately stifle real innovation.
Neglecting to point out the use of a surrogate marker, particularly if it hasn’t been shown to predict benefits that matter, can give a misleading impression about the reliability of a study:
- Several stories reporting on a drug called evolucumab, known as a PCSK9 inhibitor, said it dramatically lowered LDL cholesterol in a 24-week trial, but didn’t note that LDL is a surrogate marker for heart disease. A source told HealthNewsReview.com blogger Alan Cassels that the trial “proves nothing about benefit because there are no long-term studies of PCSK9 inhibitors’ effect on cardiovascular outcomes.” Wrote Cassels: “In other words, the drug might lower LDL, but so what?”
- A news release claiming that blueberry concentrate improves brain function in older people failed to point out that brain blood flow and other biomarkers were “not a measurable clinical benefit,” according to our review.
Some tips for evaluating surrogate endpoints
- Learn to identify likely surrogate measures. Writes Hilda Bastian: “One rule of thumb is, if you need a laboratory test of some kind, it’s more likely to be a surrogate. Whereas symptoms … or harm caused by a disease, are the direct outcomes of interest.” Often those meaningful clinical endpoints are referred to “patient-relevant outcomes,” she notes.
- Find out whether a surrogate has been validated. In some cases, follow-up trials have established the accuracy of a surrogate at predicting outcomes that matter. But for every treatment, that association needs to be confirmed. There is high-quality evidence that blood tests measuring LDL-C — an estimate of the average amount of cholesterol in a person’s LDL particles — is a reliable surrogate for assessing the ability of statins to lower heart disease risk. But other types of drugs that lower cholesterol, such as niacin, do not appear to have the same benefits.
- Be wary of “breakthroughs” based on unproven surrogates. Pricey and potentially harmful drugs for hepatitis C known as direct-acting antivirals won FDA approval via short-term studies measuring their ability to reduce the amount of virus in the bloodstream, known as “sustained virologic response” or SVR. Backers of those drugs contend SVR is reliable because it has correlated with fewer deaths in an older generation of treatments as well as other health benefits, but some medical experts aren’t convinced. Wrote HealthNewsReview.org Managing Editor Kevin Lomangino: “We have yet to see a story on a clinical trial showing that the drugs helped anyone live longer or even that they reduced rates of complications like cirrhosis or liver cancer.” A Cochrane evidence review concluded it’s “questionable” whether SVR “has any clinical relevance” to people with hepatitis C.
- Be aware of the use of surrogate measures in assessing side effects. Reliance on measures of bone loss rather than actual fractures may have overestimated the risks of contraceptive Depo-Provera on bone health.
- Know that surrogate measures can mask harms. Harmful events are often rare and may take longer to emerge than benefits, so long-term studies are crucial to assess their impact.
- Don’t assume that because a drug has been available for years, it must work. Regulators rarely withdraw ineffective drugs, so even those approved with evidence based on surrogate measures can remain available despite follow-up trials that reveal they’re no better than a placebo. Often rigorous follow-ups are never done.
- Look for surrogate measures in trials that compare treatments. The ALLHAT blood pressure study compared an older treatment — thiazide diuretic — with a set of newer and more expensive drugs. All had performed similarly in lowering blood pressure but the tried-and-true diuretic was ultimately more successful at preventing fatal heart attacks.
- Don’t fall prey to misleading jargon. “We have begun to use terminology that allows people to misinterpret what we’re saying,” Leonard Saltz, chief of gastrointestinal oncology at Memorial Sloan Kettering Cancer Center, told Medscape. He argues that progression free survival, which has nothing to do with a patient’s actual survival, should be called “the progression-free interval.” Yet he said researchers “very comfortably allow the audience listening to us—whether it’s our patients or a larger audience of professionals—to misconstrue that we’re talking about survival.”
Addendum: Some of us understand things better with a dose of humor. James McCormack, PharmD, provides it in this video, “The Surrogate Battle: Is lower always better?“