Read Original Story

More discussion of study’s limitations needed in NPR’s story on chronic fatigue test

Rating

3 Star

Categories

Scientists Edge Closer To Elusive Lab Test For 'Chronic Fatigue Syndrome’

Our Review Summary

This story about the potential use of cytokine testing to diagnose and then guide the treatment of chronic fatigue is cautiously worded. It reports that a new study “could point the way.”

Yet even with such careful language, readers are likely to get an inflated sense of what the latest study actually reported, because the story seems to assume that high levels of cytokines are causing CFS/ME and that tracking these levels will help guide treatment. However, the study under discussion, which is observational in nature, is incapable of proving such a relationship. As such, it’s too early to suggest that these lab markers can be used to diagnose the condition or monitor treatment — at least not without the inclusion of strong qualifying language.

 

Why This Matters

Back in 2015, a flurry of stories announced “A Test For Chronic Fatigue Syndrome”, “Researchers are hopeful that an immune test could help improve diagnosis”, “A Test for Chronic Fatigue Syndrome, and a New Hint at Its Origins” and so on. Helping readers clearly understand that the latest study is still fairly preliminary is important.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The story talks about work to develop a commercially available test, but there is no discussion of what such a test might cost. A story needs to at a minimum discuss what is known about the costs of diagnosing and potentially treating a cytokine-based condition.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Satisfactory

The most relevant text here is: “Out of 51 cytokines investigated via sophisticated fluorescence-based testing, only two of the cytokines differed, in their total concentrations, between the ME/CFS and control groups. But, levels of 17 of the cytokines varied dramatically between the patients with mild versus severe ME/CFS symptoms. Of those 17 cytokines, 13 were types that promote inflammation.”

The issue is, though, what does this actually mean in terms of diagnosing a patient as having ME-CFS and treating them with a potentially risky drug such as rituximab that is mentioned in this piece. While the story could have done a better job on that point, we think the discussion here is sufficient for a satisfactory grade. We’ll address the broader implications and the story’s shortcomings below under the evidence quality criterion.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

There is no discussion of potential harms of testing. Notably, since the researchers noted broad overlaps between cases and healthy controls in the levels of almost all the cytokines measured, it seems likely that test results could be easily misinterpreted to either label healthy people as sick or sick people as health.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

In short, the story seems to assume that high levels of cytokines are causing CFS/ME and that tracking these levels will help guide treatment. However, the study under discussion, which is observational in nature, is incapable of proving such a relationship. As such, it’s too early to suggest that these lab markers can be used to diagnose the condition or monitor treatment — at least not without the inclusion of strong qualifying language. Moreover, it’s unclear based on current evidence how accurate any test based on this research would be and therefore how useful in the clinic. Highlighting these and other limitations would have helped the story make clear to readers just how much work remains.

Does the story commit disease-mongering?

Satisfactory

The story does not commit disease-mongering. It would have been helpful to explain that fatigue is a very common symptom and that most patients with fatigue–even prolonged fatigue–won’t be considered as having ME-CFS.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story reports that the researchers are “working on developing a panel that could be used commercially,” thus at least implying that they stand to financially or professionally benefit from positive research results.

The story includes comments from one of the reviewers of the study, but should have noted that the reviewer and the lead author of the study also collaborate on other research. This connection was clearly noted in the footnotes of the journal article.

The story would have been better if it had included comments from at least one expert not closely associated with the authors of this journal article, and more strongly pointed out the potential conflict of interest.

Does the story compare the new approach with existing alternatives?

Satisfactory

The story reports some of the possible explanations for the failure of existing tests to accurately diagnose chronic fatigue.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The story reports that both this type of test and proposed treatments are still experimental.

Does the story establish the true novelty of the approach?

Satisfactory

The story establishes novelty in this way: “The link to gradation in severity, rather than simply seeking a positive versus negative result, represents a new approach to the search for biological markers for the illness.” And: “What the latest research shows, Komaroff tells Shots, is that “levels of many cytokines do correlate with symptoms: The higher the blood level, the worse the symptoms. That supports the theory that the cytokines are a cause of the symptoms.”

But, the story doesn’t note previous publications by the same researchers and others that also point to some of the same cytokines as potential targets for tests and treatments. A flurry of stories two years ago (examples here, here and here) appear to include broad statements that are remarkably similar to those reported in this latest story. Readers of this story do not get a clear picture of how the results of this study compare to those of earlier studies.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story includes quotes that appear to be directly from interviews with an author and a reviewer of the study.

Total Score: 6 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.