Numerous problems plague Guardian’s coverage of ‘holy grail of cancer research’

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

Costs are not mentioned.

As we’ve reported in the past, one company has marketed their similar test with a $1,000 price point.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

This story errs on the side of promoting potential benefits without providing critical context.

It includes two reckless quotes (one by a lead author, another by an independent source) which suggest that the liquid biopsy:

• “is potentially the holy grail of cancer research, to find cancers that are hard to cure at an earlier stage when they are easier to cure … potentially [this test] could be used for everybody.”
• “could unlock enormous survival gains and well as dramatic productivity benefits.”

This study did not address cure, survival, or productivity.

Furthermore, the story lists the accuracy of “diagnosing” 10 different types of cancer (with an ‘accuracy’ of 56% – 90% depending on the cancer) without making it clear these are the percentage of people already known to have cancer who had a positive test confirm its presence.

We did appreciate this inclusion:

The number of patients in who cancers were detected was small. For example, although the test detected ovarian cancer with 90% accuracy, only 10 ovarian cancers in total were detected.

To be useful to readers, the story should have included the rate of false-positives and false-negative results. Or, what’s known as test sensitivity — how often the test is correct when the disease is present; and specificity – how often the disease tests negative when the disease is not present. The percentages reported likely represent the sensitivity. So the article did provide some numbers, but it was limited and not explained in a way that would allow the reader to appreciate how good (or not good) the test really was.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The story includes no mention of what many experts consider to be the most important considerations when it comes balancing the harms vs. benefits of liquid biopsies. These include:

• For such a test to be clinically useful it would need to be both highly specific and highly sensitive (see benefits, above) — because a false positive could traumatize a patient, and a false negative could result in an untreated cancer.
• A positive result would likely lead to more invasive testing — which carries significant downstream risks and costs — and may simply identify an “early cancer” that may not develop into cancer at all.
• At least some cancers caught “early” would be overdiagnosed — meaning they grow so slowly that they would never cause a problem and are better left undetected.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story includes some basic details about the trial. But it didn’t discuss limitations–notably that any data presented at a medical conference is preliminary until published and peer-reviewed.

Also, the story touts potential benefits for screening healthy-seeming people for early signs of disease. But in the study itself, about half of the people had already been diagnosed with cancer. If the test is applied to the general population as this story suggests is appropriate, there will likely be many more false-positives than were recorded in the study.

It’s also important to note that an expert panel from ASCO and CAP (American Society of Clinical Oncology; and College of American Pathologists) recently reported:

1. There is not enough evidence at this time to support the routine use of liquid biopsies to screen for cancer, make treatment decisions, or monitor treatment progress.
2. Findings comparing liquid biopsy with tumor tissue (the current gold standard) are inconsistent. So negative liquid biopsies should be confirmed with tissue testing.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story didn’t disclose that the researchers had significant conflicts of interest. (See disclosures.)

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The current gold standard for confirming the diagnosis of most cancers is a tissue sample. This is not addressed in the article and should have been. It also could have been clearer that for some cancers there are no approved screening tests.

Does the story establish the true novelty of the approach?

Not Satisfactory

There are a host of different liquid biopsies being tested. They look for a variety of markers associated with cancers such as proteins, cancer cells, and even circulating tumor DNA (ctDNA) is in the case of this study.

This was not mentioned in the article.