Read Original Story

Blood Test May Predict Risk of Diabetes

Rating

2 Star

Categories

Blood Test May Predict Risk of Diabetes

Our Review Summary

The article discusses a lot of the evidence, and in general it’s technically accurate, even presenting finer details than in the press release. But there’s scant evaluation, and no independent sources are quoted. The content in places is eerily similar to the news release. Without evaluation of the evidence, we lose the context, and this exploratory lab research reads a lot like a clinical proof of a new screening test. (It ain’t.) The article also failed to point out that five of the investigators have filed a patent for this research. That’s another reason why including opinions from experts in the field who will not benefit financially from the results would have strengthened the article.

 

Why This Matters

An “accurate” test that identifies persons at high risk for diabetes is needed so that interventions designed to prevent the onset of diabetes can be targeted to those most in need. Unfortunately, this new research has substantial limitations: no information on “accuracy” is provided, since they didn’t look at the sensitivity, specificity, false positive, or false negative rates associated with this test. Also, there is potentially a serious statistical problem with the way the analysis was conducted. Sixty metabolics were looked at among a sample of 189 people with diabetes; this raises major concerns about a multiple testing problem and increases the probability that spurious associations were identified. Finally, this is not the first study to put forth tests to predict diabetes. It would have been more meaningful if the accuracy of this panel of amino acids was compared to other combinations of biometric tests that have been shown to predict diabetes. In evaluating any study, it’s important to consider the limitations. Doing so with this research would have produced a measured view of its prospects — an optimistic one, but realistic. A new screening test is not a foregone conclusion.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Applicable

It’s a bit early for costs, since no clinical test has been devised. When we get there we can start asking how expensive this type of metabolite profiling is and the costs and cost-effectiveness of screening people, especially the asymptomatic high-risk folks with normal blood sugar identified in this study.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory
The benefits of this research as a step on the path to new screenings or therapies are not adequately quantified. Nor does the story distinguish between potential benefits for patients and interesting results that were found in the lab.

The claim high in the story that ‘blood tests that screened for these amino acids accurately predicted risk of type 2 diabetes’ sounds a little too premature. This wasn’t a screening test that reaped any results from patients. It was lab research that is several steps behind any actual screening test. This may be why neither the study nor news release use the words screen or screening. Accuracy is another concept not raised, which is unfortunate.The accuracy and precision of the test were not assessed, and, in fact, when the researchers tried to see how predictive these amino acids were in other groups of people (known as replication cohorts), 1 of the 5 amino acids no longer had a significant association.

The researcher’s press release quote promotes the idea that in the future we could have a screening test that could identify people at high risk for diabetes, which would lead to early intervention. That would be a nice benefit. Unfortunately this research does not say it’s possible. What if, by the time you have elevated amino acids, it’s too late? What if these amino acids reflect genetic or largely nonmodifiable risk factors for diabetes? The story has arrived early to a party that’s not on the calendar yet.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Yes, Virginia, screening tests do have potential harms. If we’re ready to raise the possibility of benefits of a hypothetical test, we need to raise the equally unknown possibilities of false positives and false negatives. The authors do not address the risk that follows from misclassification of non-high-risk patients as high risk. These risks include exposure to unnecessary medical interventions, difficulties getting insurance, and other potential impacts on the way they plan their lives.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory
When we evaluate the quality of evidence, we can give readers a guidepost to how far along the research really is. This is a tantalizing result that generates ideas about how we might predict diabetes and new pathways to perhaps prevent it, but this study is not going to change practice tomorrow. A more critical analysis was needed here to make that clear. The story does include the number of subjects, follow-up period, and case-control process. It also says larger studies are needed to confirm the results. What’s chiefly missing is an evaluation of the design. This wasn’t a clinical trial of a screening test. The authors conducted a matched case control study to look for patterns. How many people does this test miss? How many does it falsely identify? Can it be truly generalized to a large population, given that most subjects were white and of European descent? Additionally, as mentioned above, screening over 60 metabolites in a small sample creates a statistical situation where false associations are more likely to show up.

Does the story commit disease-mongering?

Satisfactory
The article restrains from calling this surrogate measurement a diagnosis of diabetes and makes the point, via the press release quote and other language, that the test may indicate increased diabetes risk as opposed to predestined diabetes. It would have been nice to include a bit of explanation on the difference between diabetes and diabetes risk, as we’re not sure some readers will understand the chronology in packed phrases like “an early sign of type 2 diabetes risk” or “future diabetes risk.”

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

No independent sources are cited, and the study discloses that five of the investigators have filed a patent application “pertaining to metabolite predictors of diabetes.” This is crucial information for readers that was missed by the story.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

There’s no new approach yet, but, the story does present this research as a step toward a new type of screening. The headline says, “Blood Test May Predict Risk of Diabetes.” With that in mind, we think important contextual informatoin was missed. There should have been some comparison to other biometric models that are in the literature and used to identify patients at high risk for developing diabetes. Also, it would have been good to put this new evidence in context with what doctors already know. Many biomarkers are known to be predictors of diabetes. What you would like to know is once these other factors have been taken into consideration, how did this amino acid screen improve on those techniques?

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The article says that further studies will be needed. It would have been useful to state that any clinically available screening test would be years off. The article doesn’t give readers a sense of how far away this hypothetical test is from practice. Are the tools even being used in blood labs, or is this all being done purely in a research setting?

Does the story establish the true novelty of the approach?

Satisfactory

We give the story partial credit here. It notes that several of these amino acids have also been associated with diabetes in prior studies. It does not highlight the novelty of the new study, but suggests that the novelty lies in the particular groupings of three or five amino acids.

Does the story appear to rely solely or largely on a news release?

Not Satisfactory

It was difficult to find information in the story that was not found in the news release. We give the story credit for at least acknowledging that its one quote came from the news release.  But overall we think this story leaned too hard on the release. For example, compare these eerie similarities between the story and the news release:


WebMD Article: In addition, in people closely matched for traditional risk factors for type 2 diabetes, such as obesity or insulin resistance, those with the highest levels of the three most predictive amino acids — isoleucine, phenylalamine, and tyrosine — had a five to seven times higher risk of developing type 2 diabetes than those with the lowest levels.

News release: Overall, in people closely matched for traditional risk factors for type 2 diabetes, those with the highest levels of the three most predictive amino acids had a four to five times greater risk of developing diabetes than did those with the lowest levels.

Note that the key difference is the numbers. Perhaps the author did not see “four to five times” explicitly in the study itself. We couldn’t find it either. We did find, as the story’s author did, the “five to seven times” result. That result came from the first half of the study, the derivation sample. We wonder if it would have been more balanced to summarize another way. The study itself summarizes the overall results as showing a risk “at least 4 fold higher.”

Total Score: 3 of 9 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.