Health News Review

Two women with breast cancer, trained in how to evaluate medical evidence, thought this story wasn’t as good as the Associated Press or New York Times stories.  It is vital to see news stories as people most affected see them.

Our Review Summary

Among our reviewers’ biggest concerns about this story were:

  • the lack of a truly independent source
  • and, related to that, the lack of any independent evaluation of the quality of the evidence
  • the failure to point out anything about the demographics of the women studied – and that there were significant age-specific issues that could/should have been highlighted.

Why This Matters

Women with breast cancer need facts and data to make good decisions about breast cancer treatment. Including phrases such as “a dream come true” inflates the true benefit of this research.


Criteria

Satisfactory

Does the story adequately discuss the costs of the intervention?

Satisfactory

The story did a good job discussing costs.

Not Satisfactory

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

This story was reviewed by two breast cancer survivors, both trained in how to evaluate evidence by the National Breast Cancer Coalition’s Project LEAD training.  In addition, one earned her PhD in epidemiology. Here’s how they reacted:

Benefits were confused from the beginning of the article.  Assuming all breast cancer patients have the same profile and benefit is misinformation. For example, the Young Survival Coalition sent out a statement to members that pointed out:

Overall, the patient population in this study tended to have low nodal involvement and most participants were over the age of 45 at time of initial diagnosis.  53% of the 10-year group were node negative and 54% of the five-year group were node negative.  Only 16% of both groups had four or more nodes involved.  Also, only 19% of the 10-year group and 18% of the five-year group were under age 45 at time of diagnosis.  In both the five and 10-year group, 89% of study participants were postmenopausal at time of their entry into the ATLAS trial.  In the five-year and 10-year groups , 47% and 48% of patients respectively had tumor sizes ranging from 1 to 20 mm (equivalent to 2 cm or under), while 39% and 38% respectively had tumor sizes between 21-50 mm (2 to 5 cm).

Women with breast cancer look for this kind of detail in order to make the findings meaningful in their lives.  There are ways to summarize this data in a way even in a daily news story.

Allowing the study author to use the word “cure” in an unchallenged manner is troublesome.

A major omission from this story was survival benefits were not seen until years AFTER tamoxifen treatment had ended (rather than during treatment).

Satisfactory

Does the story adequately explain/quantify the harms of the intervention?

Satisfactory

Mixed bag – but we’ll give the story the benefit of the doubt.

Good points:

  • The story presented some of the harms clearly, including one patient’s viewpoints on the personal, physical and emotional costs of continuing tamoxifen (e.g., severe hot flashes, dry eyes, joint pain, weight gain, “dryness that made sex unbearably painful”)
  • It reported “doubling the length of treament also doubled the risk of endometrial cancer”
  • It said “doctors saw no increase in strokes, which has long been a concern with tamoxifen.

Room for improvement:

The story provided only one woman’s tale of side effects.  It provided no estimate of how often these occur. And the Young Survival Coalition pointed out another age-specific concern in the statement it released:

Finally, a particular concern with this study is the impact that this study may have on young women’s family planning.  Many young women already wait until they have completed their five-year course of Tamoxifen before trying to start a family.  Will this study cause that delay to become 10 years?  And, are the modest benefits shown in this study worth that?  That is a very personal decision and again, we recommend speaking with your physician.

Not Satisfactory

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

There was no independent analysis of the quality of the evidence.  Quoting a researcher who led tamoxifen’s development saying “This is a dream come true for women.  It’s very exciting” doesn’t amount to an evaluation of the evidence.

In addition, the two women with breast cancer who reviewed this story noted that women who are interested in learning more were not even told that this study was published in the Lancet.  They were only told that it was presented at the San Antonio Breast Cancer Symposium.  Both the competing NY Times and AP stories – which we also reviewed – mentioned that the story was published in the Lancet.  So interested parties who read those stories would know where to go to find more information.

Satisfactory

Does the story commit disease-mongering?

Satisfactory

There is no evidence of disease-mongering.

Not Satisfactory

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

THIS IS A CORRECTED COMMENT IN RESPONSE TO A COMPLAINT BY THE REPORTER OF THE STORY – HER COMMENTS IN THE COMMENTS SECTION BELOW.

WE INITIALLY SAID THERE WAS NO QUOTE FROM ANY INDEPENDENT EXPERT NOT CONNECTED TO THE STUDY.  THAT WAS INCORRECT.

HOWEVER, OUR CRITERION IN THIS CATEGORY READS: “In order to get a satisfactory score on this criterion, two things must occur: There must be an independent expert source quoted (someone not directly connected with the research) and there must be some attempt to let the public know about potential conflicts of interest.”

AND WE POST MORE BACKGROUND ON THIS CRITERION HERE.

This story DID NOT include reference to partial funding of the study by AstraZeneca, which produces a brand of tamoxifen. The competing stories by the NY Times and AP did report on AstraZeneca funding. AP went further in pointing out that the study leader “has been a paid speaker for a company that makes one of the drugs.”    So the score remains unsatisfactory despite our initial error (corrected) on the first point.

The interview with the administrative director of the Johns Hopkins Cancer Survivorhip Program about her own experience with tamoxifen use added important perspectives. (The reporter apparently missed that we referred to this in a positive note all along.)

Not Satisfactory

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The two women with breast cancer who reviewed this story found one good point, one weak point in the story on this criterion:

  • Good: Aromatase Inhibitors were briefly discussed.
  • Weak: One sentence about tamoxifen as the “only hormonal option for women before menopause” is confusing, even inaccurate.
Satisfactory

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

Availability is well-established in the story.

Satisfactory

Does the story appear to rely solely or largely on a news release?

Satisfactory

It’s clear that the story did not rely on a news release.

Total Score: 6 of 10 Satisfactory


Comments

Liz Szabo posted on December 7, 2012 at 12:38 pm

I would appreciate it if you would correct your review, which repeats several factual errors.

You state that our story lacks an independent voice. Did you read far enough to see the quotes from Dr. Jennifer Litton of M.D. Anderson? What about breast cancer survivor and nurse Lillie Shockney. director of the survivor program at Johns Hopkins? Or breast cancer survivor Alicia Staley? Yes, Dr. Jordan made a glowing statement about tamoxifen, a view shared by many others. But we pointed out that 80% of women stop taking tamoxifen earlier than scheduled, and listed a very long list of side effects, including vaginal dryness that makes sex unbearable. While these patient voices were cut from the print story because of space considerations (not by me), Dr LItton appears even in the shorter version. Shockney tuck it out on five years of tamoxifen, while Staley notes that she quit early to regain some quality of life. Shockney even raises the provocative question of whether weight loss might do as much to save lives as tamoxifen. So, like the story or not, it contains an independent voice and I would appreciate your correcting the review.

I am very sensitive to the views of women with breast cancer, and that is why I included two in my story. Your review states, in several places, that I lacked an independent voice, which simply isn’t true.

Gary and I have argued in the past about which version of our stories you should review. In the past, he has claimed that reviewers only need to look at print. However, our online circulation vastly outstrips our print circulation, making this an outdated, even 19th-century sort of argument.

If you are going to review a full-length AP story, then you should, in fairness, review the full-length online version of USA TODAY stories. Why? Because newspapers and radio stations have the right to edit that AP story in any way they choose. They can cut it down to a two-paragraph brief, leaving out far more details than you think were left out of ours. Only a fraction of news customers actually run AP stories unaltered, at full length, with no additions or deletions. Clients of the NY Times may trim their news service stories for space, as well.

Thank you for addressing this promptly.

http://www.usatoday.com/story/news/nation/2012/12/04/hormone-treatment-breast-cancer/1746805/

Reply

    Gary Schwitzer posted on December 7, 2012 at 1:06 pm

    Liz,

    Thanks for your note.

    You did, indeed, have an independent source. And we will correct that part of our comments. But you did not identify the partial industry funding of the research (as both the competing NY Times and AP stories that we also reviewed did).

    Our criterion reads:

    There must be an independent expert source quoted (someone not directly connected with the research) and there must be some attempt to let the public know about potential conflicts of interest.

    You are free to disagree with our criteria – as you often do – but they are what they are.

    You may not have read our review thoroughly, but we DID comment positively about the quotes from Lillie Shockney: “The interview with the administrative director of the Johns Hopkins Cancer Survivorhip Program about her own experience with tamoxifen use added important perspectives.”

    Regarding which version of the stories to review, we review what we find online. We are very limited in our ability to see everything. I am the only person who works fulltime on this project. And our funding for the future is very much in jeopardy. So, yes, we know what it is like to be short-staffed and to have economic woes – just like journalists. Everything we review comes from online searches. That has always been the case. No surprises. No changes. Nothing to apologize for or correct in that policy. That’s the way a growing number of people find their news as you are well aware.

    I wanted to respond to this ASAP. But I need to wrap this up as I have caregiving needs for my 92-year old Dad to take care of today. And there’s no one else on the clock right now to respond further to your complaint.

    Gary Schwitzer
    Publisher

    Reply

Liz Szabo posted on December 7, 2012 at 10:08 pm

Gary, I’m sorry to hear about your dad.

When you or your assistants get a moment, though, I would appreciate it if you would correct another error. You said, “The story provided only one woman’s tale of side effects.”

Actually, we provide two: Lillie Shockney and Alicia Staley. I’m not sure how many patients you expect to be quoted in one story, but two is more than most. I’d appreciate a correction to note that we quoted TWO patient’s quotes. So, just for the record, there were five people quoted: two who’ve researched tamoxfien, two patients with very negative experiences of tamoxifen, one impartial researcher. I actually talked to three other independent researchers, but chose Litton as representative. Overall, the quotes balance one another out and even provide an ironic counterpoint.

As you know, we are not a medical journal and we can’t possibly address every detail of a study. Even when we do, reporters are not always in control of what ends up in the final draft. The placement of a story, both online and in print, helps to set the tone for the story, something that your site doesn’t address. Placing a story on A1 provides an entirely different context than playing it deep inside the D section, where this story ran.

Your reviews are quick to criticize. Maybe too quick.

Thanks for making this additional correction.

Reply

    Gary Schwitzer posted on December 8, 2012 at 9:37 am

    Liz,

    Did you realize we gave your story a SATISFACTORY score on the criterion in question – our “harms” criterion? Because your comments resonate as if we slammed you on this.

    It is clear that we failed to reach you with the essence of our constructive criticism – delivered while giving you a SATISFACTORY score on this criterion. And that essence is this: – one woman’s (or two womens’) stories of side effects don’t provide an estimate of how often these occur. The plural of anecdote is not data. As in my response to your earlier complaint, I point you to our statement of what our harms criterion looks for – including whether a story fails to quantify potential harms, relies too heavily on a patient anecdote, etc. You are free to disagree with this criterion, but this is the way we have applied this criterion – to more than 1,800 stories over nearly 7 years. If you think we hold the bar too high, so be it. But since you wrote in your earlier comment – “I am very sensitive to the views of women with breast cancer” – something about which I have no doubt, please don’t lose sight of the stated fact that the two reviewers of this piece were both women with breast cancer, and very smart ones, trained in the evaluation of evidence by the National Breast Cancer Coalition’s Project LEAD. If you choose to ignore their constructive criticism, that is your choice.

    Finally, I question your statement that – “The placement of a story, both online and in print, helps to set the tone for the story, something that your site doesn’t address.” When we in the general public surf for stories online, we find them amidst a sea of health care stories. If you think most of us can perceive that a tone has been set by the way in which we may have stumbled upon an online story, I think you’re mistaken.

    Liz, this online discussion seems to be going in circles. You’ve made your points – twice – and we’ve tried to make ours. We can agree to disagree. We can also discuss this by phone, or I’d be happy to come to your newsroom the next time I’m in town.

    When we recently checked the average grades we’d given 14 news organizations, your paper was rated well above the average of the group – and 5th out of 14. I have to rely on my memory because we don’t systematically track any individual’s performance – but I would bet that your own stories scored much higher than the paper’s average and would be among the top individual performers over our 7-year existence. We have a great deal of respect for your work. You clearly are a caring, hard-working journalist. When we notified you about our comparison of the NY Times, AP and USA Today stories, we wrote that “All three were good. Our reviewers thought they stacked up as good, better and best.” If you do not want to accept the constructive criticism of our reviewers – both women with breast cancer – and approved by me prior to publication, that is your choice.

    Reply

We Welcome Comments

But please note: We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, profanity or any from anyone who doesn't list what appears to be an actual email address. We will also end any thread of repetitive comments. We don't give medical advice so we won't respond to questions asking for it. Please see more on our comments policy.