Not only does this story neglect to provide any caveats regarding research abstracts presented at conferences, it omits the number of subjects in the study. One of the most important pieces of context for this study was that it only had 12 subjects. That needed to be in the article. In general, the term "artifical pancreas" builds unrealistic hope for this technology for patients, and some balance and caution would’ve enriched the optimism with realism, doing a greater service to those with diabetes who are very interested in this topic.
This story matters because hypoglycemia is dangerous and self-monitoring combined with use of insulin requires a patient to be an engaged and knowledgeable self-manager. The hope of this technology is that it would function more like a normal pancreas. But that’s it’s hope. Although the story provides little information about the risks and benefits observed in this small pilot study, it implies that it’s almost ready for prime-time, despite late hints at the end of the article that the technology is not there yet.
The discussion of the Ritholz research on continuous glucose monitoring (CGM) compliance is somewhat disconnected and only loosely germane to the artificial pancreas. Research on CGM, which is approved and in use, has clinical relevance unrelated to the investigational "artificial pancreases," of which it’s a component. We can only assume this study was covered in the article because it was discussed on the teleconference call that was a source of information for the story.
Costs of the new approach or the currently available components, continuous glucose monitors and insulin pumps, are not described. The two components are not cheap and many insurance companies aren’t covering them except in special circumstances.
We do receive one set of data on benefits, the 70% vs. 47% for time spent in target blood glucose. We’re unable to verify these numbers, as the abstract available online and MedPageToday coverage both cite "Time in Target" as 72% and 50%. Perhaps 70% and 47% come from a different source, such as the press teleconference or presentation. We cannot verify it.
Regardless, as we said for the harms, the comparative benefits are not meaningful if we don’t know who the comparator group was. Furthermore, we feel than an independent voice should’ve been provided to tell us the meaning of the benefits. According to the abstract, the key benefits were a) reduced variability and b) reduced time duration and severity of hyperglycemia. What does that mean for patients, or even doctors? Why does glucose deviation matter if, as the abstract says, the average glucose levels were the same in both groups? Just giving us "70%" next to a lower number, without an explanation or an identity for the comparator group, says zilch about the value of this research.
The investigator is quoted as saying that "We reduced hypoglycemia by half." There’s a dissonant choir of problems with this statement. First, it’s not clear from the sentence’s placement in the article if this result came in the prior study with children or in the current study with adults. (The rest of these points assume the latter.)
Second, the article doesn’t identify the comparator group (those who just used the insulin pump), so a relative result is meaningless. So when the claim is made, "We reduced hypoglycemia by half," readers should be told "half of what?"
Third, the fact that they used a relative risk reduction figure, not an absolute one, is important because this study was so short and small, using relative risk reductions masks the weight (or thinness) of the evidence, especially when we don’t have the statistical significance.
Fourth, we’re not sure what data the investigator’s statement of reducing "hypoglycemia by half" is based on. In the abstract, it’s says that the time spent below target glucose was approximately half but the difference was not statistically significant, no doubt due to the small power and short duration of the study. If our interpretation is correct, the story’s author has touted a benefit that’s insignificant, statistically speaking.
Fifth, it should’ve been clearer about what "reduced hypoglycemia" mean? The total time spent below target glucose range — which averaged out over the course of the study — is different than a (severe) hypoglycemic event, which some readers may read in this result. How dangerous is it to spend any time below the target range, if only temporary and possibly briefly? In other words, is it a harm or an efficacy issue? In the abstract, we read about 1 serious case of hypoglycemia with the insulin pump, 0 with the closed-loop system; but again, with 12 patients using each for a night, statistically we’re not sure what that tells us.
Regardless of the true intent of this statement, and the true harms that occurred in the study, an independent analysis may have clarified such confusions.
No overt disease mongering.
No sources appear to have been quoted besides those on the teleconference. We don’t know the basis for the call, but it sounds like a press briefing.
Conflicts are mentioned for the people directly involved in the research. But readers can’t know or judge for themselves whether others who were quoted in the story were truly independent sources – with no funding connection to this research.
The artificial pancreas approach isn’t compared to any other alternatives, if only because the article never identifies the comparator group as those using the insulin pump.
One the one hand, the article tells us that the technology is emerging, the algorithm is still being developed, and the whole approach is still being researched in future studies. On the other hand, the article’s tone and unbalanced quotes from investigators make the ultimate availability seem like a foregone conclusion, which it isn’t after a 12-subject study. What’s missing is at least some emphasis, ideally early, that this research is quite experimental at this point, with some discussion of the steps between this small study and the technology potentially going to market.
The article also takes until the third to last paragraph, after the discussion of the Ritholz study, to explain what an "artificial pancreas" actually is; burying that explanation and leading with an illustration of a real human pancreas does nothing but suggest to lay readers that an "artificial pancreas" is something like a prosthetic organ, instead of a computer link between extant devices. It could’ve been clearer about how the "closed loop" technology is supposed to improve on the available devices. In fact, we’re not sure that the description of the artificial pancreas as a single discrete "device" matches what was used in the study, which, according to the abstract, seems to have involved a glucose monitor, a computer, and a human being making adjustments to the patient’s insulin pump.
The end of the article explains the artificial pancreas technology, implying that the computer linkage between monitor and pump is the novel part that is still under development. However, we give this criterion an Unsatisfactory rating because a) that explanation was buried in the end, after the Ritholz study summary; b) the article never explained that the aforementioned approach is the goal, not the one used in this study, which seems to have involved manual operators adjusting the patient’s insulin pump as they slept. Little information about how the "artificial pancreas" actually works, and what it consists of physically — e.g., how big is the "device"? — is provided.
This story does not appear to rely overly on the ADA press release. However, we see no evidence of any sources used outside the cited teleconference, which seems like a press briefing. Because of our uncertainty about how much independently vetted reporting took place, we’ll rate this not applicable.