The article discusses a lot of the evidence, and in general it’s technically accurate, even presenting finer details than in the press release. But there’s scant evaluation, and no independent sources are quoted. The content in places is eerily similar to the news release. Without evaluation of the evidence, we lose the context, and this exploratory lab research reads a lot like a clinical proof of a new screening test. (It ain’t.) The article also failed to point out that five of the investigators have filed a patent for this research. That’s another reason why including opinions from experts in the field who will not benefit financially from the results would have strengthened the article.
It’s a bit early for costs, since no clinical test has been devised. When we get there we can start asking how expensive this type of metabolite profiling is and the costs and cost-effectiveness of screening people, especially the asymptomatic high-risk folks with normal blood sugar identified in this study.
The claim high in the story that ‘blood tests that screened for these amino acids accurately predicted risk of type 2 diabetes’ sounds a little too premature. This wasn’t a screening test that reaped any results from patients. It was lab research that is several steps behind any actual screening test. This may be why neither the study nor news release use the words screen or screening.Accuracy is another concept not raised, which is unfortunate.The accuracy and precision of the test were not assessed, and, in fact, when the researchers tried to see how predictive these amino acids were in other groups of people (known as replication cohorts), 1 of the 5 amino acids no longer had a significant association.
The researcher’s press release quote promotes the idea that in the future we could have a screening test that could identify people at high risk for diabetes, which would lead to early intervention. That would be a nice benefit. Unfortunately this research does not say it’s possible. What if, by the time you have elevated amino acids, it’s too late? What if these amino acids reflect genetic or largely nonmodifiable risk factors for diabetes? The story has arrived early to a party that’s not on the calendar yet.
Yes, Virginia, screening tests do have potential harms. If we’re ready to raise the possibility of benefits of a hypothetical test, we need to raise the equally unknown possibilities of false positives and false negatives. The authors do not address the risk that follows from misclassification of non-high-risk patients as high risk. These risks include exposure to unnecessary medical interventions, difficulties getting insurance, and other potential impacts on the way they plan their lives.
No independent sources are cited, and the study discloses that five of the investigators have filed a patent application “pertaining to metabolite predictors of diabetes.” This is crucial information for readers that was missed by the story.
There’s no new approach yet, but, the story does present this research as a step toward a new type of screening. The headline says, “Blood Test May Predict Risk of Diabetes.” With that in mind, we think important contextual informatoin was missed. There should have been some comparison to other biometric models that are in the literature and used to identify patients at high risk for developing diabetes. Also, it would have been good to put this new evidence in context with what doctors already know. Many biomarkers are known to be predictors of diabetes. What you would like to know is once these other factors have been taken into consideration, how did this amino acid screen improve on those techniques?
The article says that further studies will be needed. It would have been useful to state that any clinically available screening test would be years off. The article doesn’t give readers a sense of how far away this hypothetical test is from practice. Are the tools even being used in blood labs, or is this all being done purely in a research setting?
We give the story partial credit here. It notes that several of these amino acids have also been associated with diabetes in prior studies. It does not highlight the novelty of the new study, but suggests that the novelty lies in the particular groupings of three or five amino acids.
It was difficult to find information in the story that was not found in the news release. We give the story credit for at least acknowledging that its one quote came from the news release. But overall we think this story leaned too hard on the release. For example, compare these eerie similarities between the story and the news release:
Note that the key difference is the numbers. Perhaps the author did not see “four to five times” explicitly in the study itself. We couldn’t find it either. We did find, as the story’s author did, the “five to seven times” result. That result came from the first half of the study, the derivation sample. We wonder if it would have been more balanced to summarize another way. The study itself summarizes the overall results as showing a risk “at least 4 fold higher.”