Researchers raise concerns over chelation heart study

Does the story adequately discuss the costs of the intervention?

Satisfactory

The story says that chelation therapy costs about $5000 out of pocket.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Satisfactory

The story says the benefits were “very small, with 26.5% of those randomly assigned to chelation experiencing a cardiovascular problem, compared with 30% of those randomly assigned to a placebo infusion.” That’s a satisfactory handling of the statistics.

The story could have also pointed out that the primary outcome was a so-called “composite endpoint” encompassing a lot of different results, including death, heart attack, stroke, referral for cardiovascular procedures, and others. Most of the benefit attributed to chelation was due to fewer cardiovascular procedures being performed in that group, which is an important detail. Because this outcome involves some subjective determination about who should and who should not be referred for these procedures, it is considered a “softer” or less reliable outcome than rates of death or heart attack, which are outside of the investigators’ control. If chelation provided a substantial benefit, we presumably would have seen a larger difference on these other “harder” endpoints.

We’re also a little uneasy with the story’s discussion of statistical significance, which it says is “the standard for judging whether a result is real or a fluke.” It says it was “concerning” that the results of this study passed the significance bar “by only 0.001,” implying that the results would have been considered a “fluke” if the results were a few hundredths different. In fact, the results, taken at face value, suggest that there was about a 97% probability that this result could not have occurred by chance, and a change of .001 would not have altered that fact appreciably — even if it did mean that the findings would no longer be considered “statistically significant.” We think it’s a stretch to question this result — something which is almost never done in coverage of other studies that also “barely clear the bar” for significance —  without a more complete discussion of what this means.

Does the story seem to grasp the quality of the evidence?

Satisfactory

Mixed bag on this criterion, but we’ll give it the benefit of the doubt.

The story included repeated deserved criticism of the study and how it was conducted. Among other problems, the story notes that the study had high drop out rates – a factor that can introduce bias. And the story mentioned that patients had differing baseline cholesterol levels, which also could have led to a skewed finding favoring chelation.

2nd last line in the story: “All of these factors call into question the results,” says Michael Carome, deputy director of the advocacy group Public Citizen’s Health Research Group.

On the other hand, there was no warning that the findings have only been presented at a medical conference and haven’t been thoroughly peer-reviewed for publication in a journal; we may learn a lot more about how the study was conducted, and what the results mean, after the paper has been vetted by other experts as part of this process.

Does the story commit disease-mongering?

Satisfactory

The story gets a satisfactory score for this criterion.

You could argue that the study wandered into fear-mongering territory in its discussion of chelation. There’s no doubt that chelation can be a very dangerous procedure in the wrong hands, but the primary result here seems to be that it was safely administered in the context of the study. It’s not until the last sentence–and after chelation is repeatedly characterized as dangerous and possibly deadly — that the story makes this point.

Then again, this study seems so riddled with problems that it’s hard to know what to make of the results. We’ll give the benefit of the doubt here, as there’s certainly no reason to recommend chelation, and still plenty of reason to be worried about it.

Does the story use independent sources and identify conflicts of interest?

Satisfactory

Plenty of independent sources contribute to this piece. If there’s one thing we question, it’s the description of possible bias among the investigators. The story quotes a source who says that “more than half the trial sites were led by doctors who practice chelation,” which “could lead to a bias in favor of chelation.” That may well be true, but aren’t a lot of trials led by physicians and surgeons who use the treatment that they’re supposed to be studying–whether it’s a drug or surgical procedure? We agree that this is a form of bias, but it’s one that’s endemic to medical research. It feels like a double standard to flag the study for this particular shortcoming.

Does the story compare the new approach with existing alternatives?

Satisfactory

The story mentions that there are medications and lifestyle changes that clearly have been shown to reduce heart disease. That’s good enough.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

Another mixed bag for which we’ll give a satisfactory score.

The story is not clear about exactly what chelating agent was used in this study, or where one would go to find a practitioner who can prescribe it.  (But, to its credit, it mentioned problems with off-label prescribing.) It was relatively easy to find out online that EDTA was the agent used in the study.

We give the story for credit for reporting that “the Food and Drug Administration had taken the study drug off of its list of approved medications.”

Does the story establish the true novelty of the approach?

Satisfactory

This is the only large-scale trial of chelation for prevention of heart disease. The research is quite novel.

Does the story appear to rely solely or largely on a news release?

Satisfactory

There is enough original reporting for us to be sure this story wasn’t based on a press release.