Early-stage research is attractive, but there’s baggage. Preliminary studies have indeed shown an association between diabetes drugs and reduced spread of lung cancer, yet anytime we try to interpret early research, we need to keep in mind why further studies are needed.
With abysmal survival rates, lung cancer needs more options. An effective and safe—or worthwhile in the net balance—strategy to prevent, treat, or put the brakes on this deadly form of cancer would be most welcome. Thus we are eager to see future research to test the hypotheses presented in this early research.
But it’s early research, which means the results have a lot of footnotes. Not simply legal fine print, each footnote is a potential game-changer, conveying some pretty daunting uncertainties. We think it’s important that coverage of such early research address the uncertainty.
A mountain separates early research like this from more advanced clinical studies, and the mountain is mean and unforgiving. Early research comes up with new exciting ideas (it generates hypotheses). But over time we’ve found that a big proportion of these ideas don’t pan out when tested in bigger, tougher trials. They slip on the mountain and we move on to another idea. So it’s important to convey the stage of research and how many years of hard, uncertain work lie ahead before we can think about giving metformin to smokers.
It’s challenging to report on preliminary research. Less information is publicly available, which means the journalist has less to go on, and then we readers get less. But the story is new, it’s hot, it’s exciting, and there’s an awesome potential to improve a lot of lives. We understand the pressures and readers’ interest, which we share, yet maybe even just one more sentence could’ve set the tone for how early this research is.
Costs aren’t quite relevant as there are no regimens to compare yet. In other words, this was not a prospective study comparing different methods to prevent or treat cancer. However, we give a nod because we’re told that generic metformin is cheaper than TZDs.
The article presents the rates of metastatic disease for each group in the study. That said, we suggest some areas where we believe it could’ve been stronger.
We would’ve liked to have seen absolute quantities here. We would have liked to see the number of subjects in each treatment group (the “little n’s”) in addition to the study’s total headcount (the “big N”). What if 150 people were in one group and 7 in the other? It’s necessary to evaluate the quality of the study and add meaning to the 20% and 42% statistics.
Several more benefits were summarized in prose without quantities, including claims about the comparative superiority of metformin to TZDs. While space would not permit all the numbers, surely there was space for more than one.
Now, we know that the abstract itself didn’t quantify these other benefits, present the little n’s, include absolute quantities, or explain the evidence behind the trend of metformin superiority over TZDs. (By the way, the term “initial trend” suggests the result may not have been statistically significant.) While true, these facts aren’t in the abstract or ACCP statement, but we think that’s not a suitable reason for not acknowledging at least the importance of missing context– it’s instead a known peril of reporting on unpublished research. Before research goes through peer review and full publication, it has more inherent unknowns, both in the science, the vetting of quality and conflicts of interest, and the final level of information that the public gets to see. Thus, we think, when the decision is made to report on preliminary research, those unknowns need to be acknowledged to distinguish it from more mature research.
Dr. Gutterman says that more research is needed before the preventative use of metformin in smokers “could be proposed.” It balances Dr. Mazzone’s proposal of the same potential benefit; but is the cat already out of the bag since Dr. Mazzone (and Reuters) have, at this point in the article, already proposed this potential benefit? As the proposal is one with a broad audience, and broad appeal, more caveats about the status of this hypothesis would’ve been ideal.
The study may not have addressed the harms of these drugs during its chart review, but they do have well-known side effects, which could’ve been mentioned.
It gives us the number of subjects and key points about how the study was conducted. We think it could’ve used more evaluation here. Any of the following big points justify an unsatisfactory rating on this criterion.
The research was presented at a medical conference. It hasn’t been peer reviewed or published. Such research is considered preliminary and, as you’ll see later in our review, does not provide all the details necessary to evaluate it. In general, we would’ve liked to have read a discussion of the early stage of this research and the need for future studies with more subjects (only 157 here) and different designs. Dr. Gutterman only mentioned that further research was needed before we could use metformin in smokers; but more definitive research is also needed for all the other benefits described, too.
The study was not designed to assess a regimen of diabetes drugs to prevent, treat, or control the spread of lung cancer. It was a retrospective look at whether people who took these drugs for diabetes incidentally had different courses of lung cancer. It’s an intriguing result, but different types of studies, e.g., prospective ones, are needed to assess whether the approach is effective and safe in altering the development or progression of lung cancer.
The article mentions different potential effects of diabetes drugs. For instance, the first sentence states that “diabetes drugs such as metformin may help control lung cancer, and may help prevent it, U.S. researchers reported on Tuesday.” But if, as we’re told, everyone in the study had lung cancer, in whom was it prevented? The study seems to have focused on progression and survival. Other research has addressed prevention, but, based on what’s in the article and the abstract, we’re unsure it was quite accurate to suggest that this latest news did. It could very well be a simple misunderstanding of that quote from Dr. Mazzone, which is ambiguous as to which effects were observed in his study versus in prior work.
The article doesn’t engage in disease-mongering regarding lung cancer and diabetes. We liked the caveat about the success of surgical removal of early-stage lung tumors.
Is it an independent source if the quote is copied from the press release? Perhaps that’s a philosophical question. We gave it a pass on this one because we’ve already dinged the story for relying too much on a news release in the criterion above.
We’re given affiliations of the lead investigator and ACCP president. But it’s unclear from the abstract whether A) conflicts of interest were never disclosed by the researchers or B) that Dr. Mazzone disclosed that he had none. We’re not told about the other authors’ disclosures. From here we see no relevant problems, but it’s another consequence of unpublished research: once it’s peer-reviewed and published, we’ll get clear, direct, and complete information about author conflicts of interest and study funding.
As the study didn’t evaluate a particular approach for treatment or prevention — e.g., “metformin used twice-daily in people without diabetes who are at high risk for developing lung cancer” — the approach is somewhat hypothetical, and alternatives not quite as relevant at this stage. While we did appreciate the references to other, recent research on iloprost and myo-inositol in lung cancer, the brief summary of the latter research may have gone a bit too far; in April 2010 we addressed the early nature of myo-inositol research.
Embedded in the grim statistics on mortality, and the partial attribution of high mortality to progression, there is acknowledgement that our available alternatives for preventing, treating, and halting the progression of lung cancer need improvement.
It’s clear that these drugs are commonly used for diabetes and some are available generically. Ideally the article could’ve stated explicitly that they’re not approved to prevent or control cancer.
Dr. Mazzone tells us what’s unique and not unique about the study. As we pointed out under the “evidence” criterion, his quote may be the source of some confusion, but he is clear about what’s unique.
This was a judgment call. A good deal of the article seems to be derived from the ACCP statement on the study. All of its quotes, including the one from the independent source, appear in that statement. The article does attribute a metformin statistic taken verbatim from this source.
Yes, there is some reporting about context.
But we think it is wrong to lift quotes from a news release without attributing the actual source. They were not collected in independent reporting.