Clearly, in the big picture, we liked this story. It addressed most of our criteria. Our negative comments are about structure not the reporting. We found the story online. We didn’t see the print version. The sidebar could have been easily missed. (In fact, we’ve been criticized by various USA Today staffers in the past for missing sidebars entirely when we do our reviews. Online, at least, we think that’s sometimes an easy miss.) But here’s why that’s troubling in this case. The main text focuses almost exclusively on the "breakthrough" nature of the findings and speculates about how the new drug being studied might be used with other therapies to offer "rare hope" to patients with advanced melanoma. Critical details and caveats, however, are relegated to the "Questions and Answers" sidebar.
One of our reviewers wrote that this story reminded him of one of those cell phone offers that has a great monthly rate but wallops you with fees in the fine print.
We don’t mean to suggest that this story is intentionally misleading, but when the headline proclaims "breakthrough" and claims that the treatment "may prolong life" on the basis of a phase 1 study, we think that’s irresponsible. And the story’s structure further clouds the overall picture by playing up the benefits and making the caveats seem like an afterthought.
The reporting was solid – if you happened to catch all of it – presumably, as we acknowledge, a bigger issue online than in print.
* Update added August 30: We’ve been advised that now neither the online nor print versions of the story use the phrase "may prolong life." We’re told that USA Today’s online producers (not the actual copy desk) briefly used that headline. But the copy desk was told that this was wrong — and that it was explicitly contradicted by the actual story. The newspaper’s print headline was always correct, and the online versions were corrected very quickly. Our reviewers read the earlier online headline. We think it’s important to share this information with readers. Many hands in many different parts of a news organization may touch and impact a story. This is how good things can go bad – and then go good again.
The prospect of a new therapy for advanced melanoma — a deadly condition that resists almost all efforts at treatment – is, indeed, exciting. But we think it’s important not to let this excitement feed false hope for desperate patients and their families. This is a real concern when researchers claim the therapy is "the most important breakthrough in melanoma, ever" on the basis of a single phase 1 study that reported on a surrogate outcome.
This was the only one of the three competing stories we reviewed which mentioned costs. While an accurate estimate of the new drug’s costs can’t be provided at this early stage of development, the story tells us that other new cancer therapies cost $5,000 to $7,000 a month. If more stories even attempted this kind of ballpark estimate, we wouldn’t have to report that 70% of stories are ruled unsatisfactory on this criterion.
Since we already dinged this story for being insufficiently cautious in its evaluation of the evidence (a criticism that might also apply here), we’ll award a satisfactory here to recognize its generally good reporting of the numbers. It notes early on that the findings are applicable only in the estimated 50% of advanced melanoma patients with a specific gene mutation. And it provides the absolute number of patients who responded to therapy and compares this with the responses typically seen with other drugs. Again, we wish the story had specified the limitations of a small, uncontrolled study that assessed an outcome (tumor shrinkage) that might not reflect longer life or better quality of life.
A decent effort here that ultimately did not meet our standard. The story notes that the side effect profile of the drug is relatively benign compared to chemo and lists common adverse events associated with the drug such as rashes and joint pain. It added a clarifying comment from an expert who said that even these "mild" side effects can become bothersome over time–a nice touch.
Still, we think it’s unsatisfactory that the story made no attempt to quantify these harms, especially the sharp increase in squamous-cell carcinoma associated with the treatment. In our view, any treatment that increases risk of cancer — even a highly treatable cancer — has to be viewed very cautiously. The squamous cell cancers are called "non-lethal" but we don’t know how the treatment itself could modify their behavior and it is conceivable they could be more aggressive or difficult to treat. In addition, could the drug also promote other cancers that aren’t showing up in this very small sample of patients?
Perhaps this story’s sidebar should have answered the questions: "What is a Phase 1 clinical trial designed to evaluate" and "What are the different types of cancer clinical trials?" This story and its competitors seem a bit confused on this point.
Although the story does a reasonable job of summarizing the details of the research and the outcomes the study authors reported, it got swept up a bit in the excitement surrounding the new study and failed to address some key issues. Most notably, the story did not call attention to the fact that this small, uncontrolled study cannot tell us whether the new drug improves overall patient survival compared with standard care. So the headline’s suggestion that the drug "may prolong survival" seems quite premature. The story makes much of the fact that tumors shrank and were "kept in check" by the new drug, but tumor shrinkage does not always translate to longer life in cancer studies. Another issue not explicitly addressed is that most patients’ cancers developed resistance to the drug and eventually resumed growing. Although the story does state that the drug is "not a cure for melanoma," we think the story should have included some direct comment on the fact that the drug’s effects, for now, appear to be short-lived for many patients.
This story did not exaggerate the consequences of advanced melanoma.
A borderline satisfactory. While the sidebar notes that the study was funded by drug developer Roche, we wish it had commented a bit more on the extensive links between the study authors–several of whom are quoted in the story–and the drug industry. Notably, lead author Keith Flaherty reported receiving consulting fees from Roche, and several of the study coauthors were employees of Roche or its partner, Plexxikon. This story did include comments from two experts who were not affiliated with the study.
The story discusses the ineffectiveness of currently approved treatments for advanced melanoma and mentions another experimental drug that has shown encouraging results in investigational studies.
The story calls the new drug experimental but notes that there are trials recruiting advanced melanoma patients where treatment with the drug might be available. However, it could have cautioned that many patients are deemed ineligible for these trials as the inclusion and exclusion criteria are often very strict. The story also notes that Roche plans to apply for approval of the drug in 2011 "if additional studies are positive." We wish the story had included a caveat about the many obstacles that might affect this timeline.
A close one here, but we’ll err on the side of generosity since the story provided more information than competitors.
While the results do appear to herald a potentially important advance in melanoma treatment, we think it’s too early to conclude that this therapy is "the most important breakthrough in melanoma, ever," as one researcher affiliated with the study breathlessly put it. This enthusiasm might have been acceptable had some more cautious voices been included to balance things out.
This story was clearly not based on a news release.