Note to our followers: Our nearly 13-year run of daily publication of new content on HealthNewsReview.org came to a close at the end of 2018. Publisher Gary Schwitzer and other contributors may post new articles periodically. But all of the 6,000+ articles we have published contain lessons to help you improve your critical thinking about health care interventions. And those will be still be alive on the site for a couple of years.
Read Original Story

WSJ reporter claims a carotid artery scan ‘saved’ his life — but there’s no way to know that

Rating

2 Star

Tags

The Scan That Saved My Life

Our Review Summary

The story about a reporter’s decision to treat a carotid artery blockage exemplifies the powerful strengths and weaknesses of first-person health care reporting. It carries a powerful emotional punch and an apparently happy ending that overwhelms the evidence presented. Numerous psychology studies document the power of “outcome bias,” in which the results of a decision affect beliefs about whether the decision was correct or not. We are very happy that all went well, but it is reasonable to assume this story would have looked very different if had not.

There is no mention of the costs of the surgical procedure. Neither is carotid stenting mentioned as an alternative to endarterectomy surgery. Discussion of carotid artery screening of people who do not have symptoms is mixed up with the reporter’s experience with testing done in order to diagnose the cause of symptoms – a somewhat common flaw of conflating diagnostic testing with screening tests. The risk that the surgery could trigger a stroke, including a fatal one, is highlighted, but the risk statistics reported here are lower than those published in major summaries of the evidence.

 

Why This Matters

The headline of the story proclaims that a carotid artery scan “Saved My Life.” That may be true, but it is also unknowable. What the best summaries of the evidence conclude is that in cases similar to those of the reporter, surgery to clear a nearly-blocked carotid artery lowers the long-term risk of stroke and death. However, stories on this topic should highlight the fact that the procedure does not actually “save the lives” of most patients. As a systematic review of the evidence by the Cochrane Collaboration concluded, using a statistical analysis known as the number needed to treat (NNT), six patients need to be treated to prevent one stroke or death within five years of the surgery. That means that for five out of six patients, the procedure probably does not alter the outcome: Either they would not have had a stroke or died even without surgery or they still had a stroke or died despite having the surgery.

Another important concern with this story is that the reporter uses his own case — where the scan was part of an attempt to track down the cause of his neurological symptoms — to argue for scanning of all older adults, including those who have no symptoms at all. This is a misleading argument since people with blockage severe enough to cause symptoms are likely different in important ways (and more likely to benefit from this scan) than those who don’t display symptoms. While it is certainly reasonable to choose a treatment shown to reduce the risk of stroke and death, it is imperative that stories clearly communicate the difference between screening and diagnosis, and the uncertainty about whether any particular patient will benefit.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

While the story reports the cost of scanning carotid arteries to be about $70, it does not discuss the cost of surgery. One analysis of Medicare reimbursements done a few years ago reported hospitals were paid about $11,000 for each patient getting a carotid endarterectomy. An earlier report estimated that in 2008 hospitals charged about $35,000 for the surgery and associated treatment, though the ultimate reimbursements by Medicare were much lower.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story does report on the results of a major trial from 20 years ago that concluded patients who were randomized to surgery had about a 5 percent risk of stroke or death within three years, “less than half that of people treated only with drugs.” But this statistic is overwhelmed by the headline (The Scan that Saved My Life) and the punchline, “mostly I feel lucky to have gotten a test that may have saved my life.”

A more recent review of the evidence by the international Cochrane Collaboration concludes that the “number needed to treat‘ (for the type of blockage and symptoms the reporter had) would be six patients. Meaning, six people have to be treated to prevent one stroke or death over a five-year period. In other words, the surgery likely does not alter the outcomes of five out of six patients. Either they wouldn’t have had a stroke or died even without treatment or they did have a stroke or died despite undergoing surgery.

Does the story adequately explain/quantify the harms of the intervention?

Satisfactory

We will rate the story as satisfactory because it does clearly point out that the surgery is hazardous, sometimes triggering a stroke, and that some patients die soon after the procedure. However, the story reports stroke and death rates at the lower end of the ranges reported in medical literature. The story says a major task force report estimated the risk of stroke and death within 30 days after surgery as less than 3 percent overall, but up to 5 percent at some hospitals. However, the lower figure in that task force report uses data only from clinical trials, which typically involve highly skilled surgeons and have better outcomes than typical clinical care. That report also said that some observational data indicates a risk of stroke and death of almost 7 percent at some hospitals. The evidence review mentioned above concludes that the 30-day risk of stroke and death is 7 percent in clinical trials of carotid endarterectomy.

The story does warn readers that few hospitals and surgeons publish their individual results.

The story would have been stronger had it mentioned that there are harms that can occur if we started screening anyone, regardless of whether they had symptoms. As the USPSTF explains, “For the general primary care population, the magnitude of benefit is small to none. Adequate evidence indicates that both the testing strategy for carotid artery stenosis and treatment with [surgery or stenting] can cause serious harms.”

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story does refer to certain major trials and guideline reports. However, the story seems to cherry-pick conclusions, favoring those that tilt toward screening and treatment while questioning those that recommend against widespread screening. It features a guideline in favor of screening from a professional society of surgeons. It does not mention the evidence review by the international Cochrane Collaboration. It also does not mention important limitations of studies that appear to favor screening — including that the medical therapy used in some of these older studies is no longer considered state of the art. The headline and punchline of the story portray a sunny and one-sided view that dismisses the complexity and uncertainty of the available evidence.

Does the story commit disease-mongering?

Not Satisfactory

Although the story accurately reports statistics on the number of people who have strokes, it confuses readers by referring to “screening” (which is testing people who don’t have symptoms) with the testing the reporter underwent to diagnose the cause of specific symptoms. After noting that a U.S. task force (the US Preventive Services Task Force) recommends against screening people without symptoms, the story says, “My experience, along with some evidence from screening of thousands of individuals, raises questions about the task force’s conclusions.”

Since his experience was with diagnostic testing, it is not relevant to a discussion of screening tests.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

It is an inherent weakness of this sort of first-person reporting that there is an absence of independent sources. All but one of the physicians quoted in the story were personally involved in the reporter’s case. The one exception is a vascular surgeon who strongly advocates for broader carotid artery screening. There are no quotes from experts who were members of the task force that recommended against widespread screening.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The story makes no mention of carotid artery stenting. This omission is surprising, since the reporter has covered this alternative on several occasions:

(Full text versions of some stories are behind a paywall.)

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

It is clear that carotid endarterectomy is widely available.

Does the story establish the true novelty of the approach?

Satisfactory

The story makes clear that the procedure has been in use for decades. What could have used more exploration–from independent sources–is whether it’s overused.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story is based on the reporter’s personal experience.

Total Score: 4 of 10 Satisfactory

Comments (2)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Thomas M. Burton

September 10, 2018 at 3:17 pm

I find it amazing that a site that purports to grade journalism didn’t take the most basic step in journalism — that of calling the subject of your report, namely me. You’d be amazed what you might have learned by doing so.
Also unfortunate is your mistaken statement that our article failed to evaluate the costs and benefits of the “intervention” or “treatment” we wrote about. You’re wrong — and you’re writing about the wrong intervention. The only step the article endorses at all is that older Americans consider carotid screening. We did not endorse carotid surgery. So, when you criticize the article for what it says, and doesn’t, about surgery and alternatives, you’re grading the wrong thing. Maybe you should fix your report card.
You suggest that our article doesn’t “seem to grasp the quality of the evidence.” Really? What evidence exactly are you referring to that we didn’t grasp? You cite a number from Cochrane about my risk of death and cavalierly describe that risk as small. But I don’t believe your criticisms take into account anything about the risk of someone in my situation winding up horribly disabled in a nursing home. I took it very seriously.
As far as the contention that my experience was “only” with diagnostic testing, and that “it is not relevant to a discussion of screening tests,” that’s wrong too. If you had picked up the phone and asked, you would understand why it was very relevant: I don’t believe any of my doctors recognized the rare symptom called limb-shaking TIA at the time I got my carotid ultrasound. So I was in exactly the same position as the scores of thousands of asymptomatic patients who get life-altering strokes without any warning. I was just lucky to have persistent doctors who pushed for more tests. If I had had a carotid screening at age 65 through Medicare, I might well have avoided surgery — and saved a lot of money. Clearly, the numbers show that thousands of people, oblivious to having a major stroke risk, are not so lucky.
I know that screening is a controversial issue, including carotid screening. But it’s a shame we never had a nuanced discussion on the topic before you issued your “report card.” It’s something good journalists do before they write.
Thomas M. Burton
Wall Street Journal

Reply

    Joy Victory

    September 11, 2018 at 9:41 am

    Thank you for your feedback.

    We do not call journalists as part of the review process. Any piece of health care journalism has to stand on its own merits, and shouldn’t require the author to explain in a phone call why certain conclusions were or weren’t drawn. That reasoning should be transparent to readers of the story, and should certainly be evident to a team of journalists and health care experts applying our 10 criteria.

    From what we read in your story, you sought out treatment for worrisome symptoms of unknown origin and were being worked up for a diagnosis based on symptoms, and so your case is fundamentally different from an older adult with no symptoms (who would receive “screening” and not a diagnostic test). We have no way of knowing what may or may not have happened had you never had symptoms and gotten screened at age 65, or whether your situation is really comparable to that of “thousands of asymptomatic patients who get life-altering strokes without so much as a warning.”

    In addition to discussing carotid artery scans, the piece makes claims about the effectiveness of carotid artery surgery: “The effectiveness of carotid-artery surgery in people without symptoms—but major blockage—was established two decades ago by the Asymptomatic Carotid Artery Stenosis trial, funded by the National Institutes of Health.” So, according to our review criteria, the costs and benefits of surgery should have been explained, as well. As for the evidence, we feel the piece should have included the findings from the Cochrane analysis that we cite in our review.

    Reply