Artificial Pancreas Continues to Show Promise

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

Costs of the new approach or the currently available components, continuous glucose monitors and insulin pumps, are not described. The two components are not cheap and many insurance companies aren’t covering them except in special circumstances.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

We do receive one set of data on benefits, the 70% vs. 47% for time spent in target blood glucose. We’re unable to verify these numbers, as the abstract available online and MedPageToday coverage both cite "Time in Target" as 72% and 50%. Perhaps 70% and 47% come from a different source, such as the press teleconference or presentation. We cannot verify it.

Regardless, as we said for the harms, the comparative benefits are not meaningful if we don’t know who the comparator group was. Furthermore, we feel than an independent voice should’ve been provided to tell us the meaning of the benefits. According to the abstract, the key benefits were a) reduced variability and b) reduced time duration and severity of hyperglycemia. What does that mean for patients, or even doctors? Why does glucose deviation matter if, as the abstract says, the average glucose levels were the same in both groups? Just giving us "70%" next to a lower number, without an explanation or an identity for the comparator group, says zilch about the value of this research.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The investigator is quoted as saying that "We reduced hypoglycemia by half." There’s a dissonant choir of problems with this statement. First, it’s not clear from the sentence’s placement in the article if this result came in the prior study with children or in the current study with adults. (The rest of these points assume the latter.)

Second, the article doesn’t identify the comparator group (those who just used the insulin pump), so a relative result is meaningless. So when the claim is made, "We reduced hypoglycemia by half," readers should be told "half of what?"

Third, the fact that they used a relative risk reduction figure, not an absolute one, is important because this study was so short and small, using relative risk reductions masks the weight (or thinness) of the evidence, especially when we don’t have the statistical significance.

Fourth, we’re not sure what data the investigator’s statement of reducing "hypoglycemia by half" is based on. In the abstract, it’s says that the time spent below target glucose was approximately half but the difference was not statistically significant, no doubt due to the small power and short duration of the study. If our interpretation is correct, the story’s author has touted a benefit that’s insignificant, statistically speaking.

Fifth, it should’ve been clearer about what "reduced hypoglycemia" mean? The total time spent below target glucose range — which averaged out over the course of the study — is different than a (severe) hypoglycemic event, which some readers may read in this result. How dangerous is it to spend any time below the target range, if only temporary and possibly briefly? In other words, is it a harm or an efficacy issue? In the abstract, we read about 1 serious case of hypoglycemia  with the insulin pump, 0 with the closed-loop system; but again, with 12 patients using each for a night, statistically we’re not sure what that tells us. 

Regardless of the true intent of this statement, and the true harms that occurred in the study, an independent analysis may have clarified such confusions.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

1. Only 12 subjects were in the study.
2. Research presented at a conference has not been peer-reviewed and is considered preliminary.
3. The study was only for 2 nights, and only for 1 night were subjects actually using the new technology (i.e., the study was extremely short-term).
4. The comparator group — the owner of the 47% result — was not simply people "not using the technology," it was the night that people used an insulin pump instead of the full-on artificial pancreas.
5. The technology used in the study wasn’t a self-contained device or automated series of devices: as we read the abstract, it required a human being to manually adjust the insulin pump over the course of the night.
6. The study focused on the overnight period, which required subjects to separately take insulin with their meal before using the device. The "artificial pancreas" didn’t replace mealtime insulin.
   

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

No sources appear to have been quoted besides those on the teleconference. We don’t know the basis for the call, but it sounds like a press briefing.

Conflicts are mentioned for the people directly involved in the research. But readers can’t know or judge for themselves whether others who were quoted in the story were truly independent sources – with no funding connection to this research.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The artificial pancreas approach isn’t compared to any other alternatives, if only because the article never identifies the comparator group as those using the insulin pump.

Does the story establish the availability of the treatment/test/product/procedure?

Not Satisfactory

One the one hand, the article tells us that the technology is emerging, the algorithm is still being developed, and the whole approach is still being researched in future studies. On the other hand, the article’s tone and unbalanced quotes from investigators make the ultimate availability seem like a foregone conclusion, which it isn’t after a 12-subject study. What’s missing is at least some emphasis, ideally early, that this research is quite experimental at this point, with some discussion of the steps between this small study and the technology potentially going to market.

The article also takes until the third to last paragraph, after the discussion of the Ritholz study, to explain what an "artificial pancreas" actually is; burying that explanation and leading with an illustration of a real human pancreas does nothing but suggest to lay readers that an "artificial pancreas" is something like a prosthetic organ, instead of a computer link between extant devices. It could’ve been clearer about how the "closed loop" technology is supposed to improve on the available devices. In fact, we’re not sure that the description of the artificial pancreas as a single discrete "device" matches what was used in the study, which, according to the abstract, seems to have involved a glucose monitor, a computer, and a human being making adjustments to the patient’s insulin pump.

Does the story establish the true novelty of the approach?

Not Satisfactory

The end of the article explains the artificial pancreas technology, implying that the computer linkage between monitor and pump is the novel part that is still under development. However, we give this criterion an Unsatisfactory rating because a) that explanation was buried in the end, after the Ritholz study summary; b) the article never explained that the aforementioned approach is the goal, not the one used in this study, which seems to have involved manual operators adjusting the patient’s insulin pump as they slept. Little information about how the "artificial pancreas" actually works, and what it consists of physically — e.g., how big is the "device"? — is provided.