Read Original Story

Survival statistic unbalances coverage of virus-based melanoma therapy


3 Star


Scientists are using herpes to treat skin cancer — and it’s working

Our Review Summary

Melanoma, skin cancer, mole.This blog post discusses the stage III trial of an Amgen treatment for melanoma, T-VEC, that is derived from the Herpes Simpex 1 virus. The modified virus does two things. It has been genetically altered to infiltrate cancer cells (but not healthy cells) where it bursts the cell membranes. The virus has also been programmed to produce a signaling chemical for the immune system to highlight the cancer location.

We’d highlight two specific suggestions for improvement in this otherwise informative story. First, the story implies that the drug is only months away from being available to patients. Though the manufacturer and authors of the study may hope this is the case, they have no way of knowing for certain. For desperate patients, such statements may lead to false hope. Second, the article provides a rosier picture of the benefit of this treatment than it should have. The appropriate overall survival statistic to report would have been the median survival rate in the entire population, 23 vs. 19 months. Though this is a statistically significant and clinically meaningful difference, it isn’t quite as dramatic as the average survival improvement of 41 vs. 21.5 months that is reported in the story. See our review comments for details on why median survival is more informative.


Why This Matters

Though melanoma represents only a small fraction of skin cancers, it is the only one that can widely spread within the body and potentially lead to the patient’s death. The primary means of curing melanoma is early identification and removal. For those with advanced disease, such as individuals in this study, the treatment options extend life but do not provide a cure for most individuals. Standard cancer chemotherapy has had little effect on advanced melanoma, but new therapies focused on boosting the patient’s own immune system have shown promise. It is with this backdrop that this study offers another immunotherapy to certain individuals with advanced melanoma. The agent is delivered by a modified herpes virus directly into the tumor itself. So a key aspect of this treatment is the ability to get at the tumor (some tumors may not be accessible) and that there are only a limited number of sites identified. Ideally one would want immunotherapies that are injected into the bloodstream and that seek out tumor cells wherever they are in the body and destroy them. As mentioned only at the end of the article, there are now other immunotherapies that more broadly target advanced melanoma, and how this new herpes immunotherapy compares is at present unknown. So while this may represent an important new option for certain patients, how it compares to other treatments and for whom should one consider it initial or salvage therapy remains to be seen.


Does the story adequately discuss the costs of the intervention?

Not Satisfactory

There is no discussion of the costs of this drug. Genetically modified drugs are often extremely expensive and because there is no discussion of alternative therapies, it is difficult to know anything about the cost, actual or comparative.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

Melanoma is a cancer on the rise and a treatment for inoperable melanoma that increases survival rate is beneficial. But we’d prefer a bit more precision in the way the statistics were handled and communicated. The 16.3 vs. 2.1% comparison is accurate, but the story could have done a better job explaining what “showing results” means and how the outcome was measured.

More problematic is the story’s description of survival benefit, which suggests an average survival of 41 vs. 22 months in the treatment vs. control groups. The more appropriate outcome to report is median overall survival. The difference for that outcome was 23 vs. 19 months. This is a statistically and clinically meaningful difference, but isn’t quite as stellar as what is reported by the story. Average survival could be misleading because it can be affected by outlier patients who do very well on the drug but are not representative of typical outcomes. Median survival is the time at which an equal number of patients do better and an equal number of patients do worse. It’s the standard outcome for such studies and what was reported in the abstract of the paper being covered.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

No mention of harmful side effects, or side effects at all, is made. The original paper does indicate that side effects were minor and that most negative effects were either unrelated or due to disease progression rather than the treatment.  However, a line indicating that most side effects were minimal would have been welcome. It would also have been reasonable to mention that virus- delivered therapies have been shown to have potential side effects that warrant caution, and that there is some worry that healthcare workers and others could be infected by these viruses. (though this one seems safe, at least from the studies performed to date).

Does the story seem to grasp the quality of the evidence?


The story does provide context regarding the limitations of the study, particularly the fact that when this trial began, there was no effective treatment for melanoma. Now there are alternative immunotherapies shown to improve survival that would be better comparators for this new agent, and the story notes that more research will be needed to see how the drug compares.

Does the story commit disease-mongering?


The article could have more quickly pointed out that this treatment is for only one cause of skin cancer, melanoma, and for those with very advanced/metastatic disease. Reading just the title and first paragraph may lead the reader to believe this is a more generic treatment for all skin cancers — most of which are locally growing and do not put one at any risk for distant spread or death. But the story does not disease monger. Melanoma is a cancer that is difficult to treat, especially in its late stages, and the scope of the problem is not exaggerated.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

Nearly every author on this paper has some association with Amgen, which is disclosed in the paper, but the blog post makes no mention of these affiliations. Only one person is quoted and he is one of the authors of the paper. This story needed the evaluation and comment of someone not involved with the phase III study. Such an unaffiliated expert would have made clear that it is uncertain when this agent should be used compared to other melanoma therapies.

Does the story compare the new approach with existing alternatives?


While the story does not compare the treatment with current standard treatments, it does explain that those comparisons were not done in the paper and that they need to be done now. Ideally this would have been communicated much higher up in the story, as this isn’t an afterthought–it is the key information to determine how this agent should be used. Otherwise, one is left with the impression that everyone with melanoma will be getting this treatment if and when it is approved by the FDA. That is far from the truth.

Does the story establish the availability of the treatment/test/product/procedure?

Not Satisfactory

The story does a bit too much crystal ball-gazing as to when this drug might become available. The start of the article refers to the drug as being months away from approval. Later in the story, the Amgen-affiliated researcher says he expects approval within the year. But his optimism might not be shared by those outside the company — the story should at least have asked for an independent take. While it’s true that an FDA advisory panel has recommended approval of the drug, the researcher can’t know if the FDA will in fact approve the drug or when. One analysis suggested that the FDA does not follow the advice of its advisory panels about a quarter of the time.

Does the story establish the true novelty of the approach?


The blog states that while some types of viral treatment, such as phage therapy, have been used in Europe and other places, this is the first time a modified virus has been used as immunotherapy against melanoma.

Does the story appear to rely solely or largely on a news release?


While the news release from the Institute of Cancer in London features quotes from the same researcher, the quotes in the story seem different enough that we can rule out excessive reliance on a news release.

Total Score: 5 of 10 Satisfactory


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.