NOTE TO READERS: When this project lost substantial funding at the end of 2018, I lost the ability to continue publishing criteria-driven news story reviews and PR news release reviews - once the bread-and-butter of the site going back to 2006. The 3,200 archived reviews, while still educational, are getting old and difficult for me to technically maintain on the back end of the website. So I am announcing that I plan to remove these reviews from the site by April 1, 2021. The blog and the toolkit - two of the most popular features on the site - will remain. If you wish to peruse the reviews before they disappear, please do so by the end of March 2021. After that date you may still be able to access them via the Internet Archive Wayback Machine - https://archive.org/web/.
Read Original Story

Drug may work in melanoma tumors in brain

Rating

2 Star

Drug may work in melanoma tumors in brain

Our Review Summary

This story offers tantalizing hints of potential benefits of an experimental drug for melanoma, ipilimumab, but without any substance that would allow readers to judge the merits of the claims. It relies on readers to know that medical meeting presentations do not go through the sort of peer review required by leading journals. It expects readers to know what a “phase II” trial is and what sort of limitations that fact implies about the value (or lack thereof) of the study conclusions. The story fails to provide context or interpretation. Indeed, the reporter did not include any comments from anyone; the story is essentially a summary of a summary.

The story highlights a trial that is claimed to be a “first;” specifically the “first to test ipilimumab in patients whose skin cancer had spread to the brain.” How significant is that step? Other than saying that brain tumors are difficult to treat, the story doesn’t explain what is newsworthy about this trial.

A more serious failure is lack of disclosure that the trials and researchers received funding from companies developing the experimental drug. The story also appears to misstate the design of the second trial that it mentions.

 

Why This Matters

Medical meeting abstracts lack context and key information on methods, patients and study limitations. Stories based on these abstracts suffer the same inadequacies.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Applicable

Although the cost of this experimental therapy may not be known, it would have been useful to point out whether this drug is similar to other drugs that carry extremely high price tags (some are thousands of dollars per dose) or might be something that might be offered at a lower cost.  

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

These trials did not compare the experimental drug ipilimumab to standard therapy. Although the story accurately transcribes what the researchers wrote about the potential use of the drug, it should have pointed out to readers that this sort of experiment is usually not designed to be able to answer questions about the relative benefits of the experimental intervention.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The story does not give any details about side effects. It is worth noting that the abstracts themselves fail to provide much details about treatment harms, which is one of the problems with basing a news story on medical meeting abstracts.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

Actually, it is difficult to assess how well this story conveys the quality of the evidence, since it is based entirely on abstracts of upcoming medical meeting presentations. By their very nature, these abstracts lack critical information about methods, patients, limitations and other key features.

The story does point out that one of the abstracts is based on a phase II trial; however it does not tell readers what a phase II trial is. It would have been helpful to note that the trial did not include any comparison to standard therapies. There is no information about what phase the second trial represented.

This story did not mention that a randomized, double-blind, multicenter trial of ipilimumab by the same group of researchers will also be presented at the same medical meeting, perhaps because the abstract details are being withheld for a few more days. Another trial of ipilimumab treatment for metastatic melanoma that has longer follow-up also was not mentioned.

It is hard to understand why the reporter chose to highlight some abstracts, but not others on the same treatment.

Also, it appears that the story may misstate the type of patient included in the second trial. The story says, “A separate study of the drug also showed signs it could work in people who first appeared not to respond to the drug.” However, the abstract itself says the trial included patients who initially did have stable disease or a partial or complete response after treatment, who were then re-treated when their melanoma showed signs of progressing. The descriptions do not match. Indeed, it appears that the reporter considered stable disease to be evidence of benefit in the first trial, but then evidence of a failure to respond in the second trial.

Does the story commit disease-mongering?

Satisfactory

There is a mismatch in this story between the specific experimental intervention and the background on the extent of the disease. The trial highlighted in the lead included only patients with melanoma that had spread to the brain… but then the story quotes total incidence and death statistics for all cases of melanoma… certainly a far larger number.  We’ll give the story the benefit of the doubt but wish this distinction had been made clear.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

Not only were there no independent sources, the reporter did not include comments from any experts at all. The story appears to be based entirely on reading presentation abstracts.

Even though disclosures about study funding and other potential conflicts of interest are posted on the same web site as the meeting abstracts, this story failed to report that both trials and the study authors received funding from Bristol-Myers Squibb, the company developing the drug, and that one of the trials was also supported by a second company, Medarex. The abstracts themselves also mention that some of the researchers are employees of Bristol-Myers Squibb or Medarex. Readers deserve to be told about these financial relationships.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The story does not mention existing treatments, nor does it mention other experimental treatments for advanced melanoma that are also being presented at this same meeting.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The story notes the drug is experimental.

Does the story establish the true novelty of the approach?

Satisfactory

The story points out that what is new about the highlighted trial is that it looks at using this experimental drug in a subset of patients, those with melanoma that has spread to the brain. However, readers might be misled by the apparently inaccurate description of the second trial in the story, as noted above.

Does the story appear to rely solely or largely on a news release?

Not Applicable

This story appears to be based entirely on abstracts posted on the asco.org web site. Although meeting abstracts are not the same as news releases, they do not provide details that are vital to providing readers with a clear understanding of both the benefits and harms of the treatment being studied.
Abstract links:

Abstract 1:

Phase II trial of ipilimumab monotherapy in melanoma patients with brain metastases.

http://abstract.asco.org/AbstView_74_50834.html

Abstract 2:

Re-induction with ipilimumab, gp100 peptide vaccine, or a combination of both from a phase III, randomized, double-blind, multicenter study of previously treated patients with unresectable stage III or IV melanoma.

http://abstract.asco.org/AbstView_74_47740.html

 

Total Score: 3 of 8 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.