This piece is a dramatic and quite moving story of what appears to be a successful trial to restore vision to functionally blind young adults with a rare form of blindness. This was a small, unblinded trial in several humans performed without any placebo or sham operation controls. The story failed to point out the weakness in drawing conclusions from such a trial. In this procedure, an altered virus containing a healthy gene that the subjects’ eyes do not possess is injected directly into the eye. Several tests of vision, both objective and subjective were performed in these subjects several weeks after the procedure. Several measures were reported to improve, but are of borderline clinical significance. Where blindness is concerned, even modest effect is worth publicizing.
What we have just described is dramatic enough. But it is an experiment – not yet a “cutting-edge therapy” as the broadcast calls it.
Significantly, the story uses interviews of inherently biased sources: the grateful patient and two investigators who hold patents on the gene therapy they are touting. It is a glaring error for the journalists not to have obtained some independent assessment of the significance of this trial. It is good entertainment, but generates too much enthusiasm. It is also unfortunate that the viewer is not provided resources for more information. The millions of viewers who suffer with limited vision might be overly encouraged by this report that has a direct effect on blindness in a miniscule minority of cases.
There was no mention of costs or insurance coverage.
This piece tugged at one’s heart strings by presenting a young blind woman who could talk about her perceived visual improvement. Notwithstanding her experience, the story failed to communicate that not every patient in this small study showed even the small improvement of the woman featured in the story. The study this story was built on reported that there was no clinically significant change in visual acuity – something this story didn’t make clear at all.
There was a small hole that formed in one eye, thought not to be clinically meaningful. There is also not enough emphasis that this is a short term (weeks to months) trial and that problems (or improvement) may well develop after a delay. The story also did not mention the past history of deaths which have been associated with some gene therapy trials.
The journalists needed to look no farther than the editorial accompanying the publication to learn that the outcome was probably not clinically meaningful, and that not all subjects enrolled in the experiments evidenced any improvement at all. The journalists accurately convey that this is a report of preliminary results of a clinical trial, but do not spell out that this is the weakest type of evidence, performed without "blinding" (that is having researchers evaluate the outcomes who are ignorant of the treatment status of the subjects) or control groups. We cannot assume, for instance, that perfomance on the walking-through-obstacles test would have remain unchanged irrespective of treatment, given the expected training effect of repetition. Similarly, there may be something about the injection itself, rather than the gene therapy, that helps the blindness.
While the story does state that these experimental subjects all had a rare form of blindness, it is not stated clearly that this procedure is not a more generalizable treatment for many kinds of congenital blindness. Nonethless we’ll give the story the benefit of the doubt on this criterion.
The greatest threat to validity of this story is that all experts cited have patents pending on this gene therapy, and this fact is noted in the source article in the New England Journal of Medicine. The story failed to mention this.
There was no discussion of treatment options for leber congenital amaurosis, though presumably there are no other treatment options.
The story does not explain that the procedure is not ready for general application. It does not make clear that this is part of an on-going clinical trial currently recruiting study subjects. Instead, it referred to the experiment as "a medical milestone" and as "a cutting-edge therapy." It’s not yet a therapy. It’s an experiment.
While making it seem like the patient profiled was the first to receive this treatment, the same group of investigators reported on a patient, Robert Johnson, last year at this time. (This was covered by the BBC and in TIME magazine). It would have been very interesting to do a follow-up story with this person to see what effect(s), if any, the treatment has had.
That said, pursuit of this approach is novel.
We can’t be sure if the story relied solely or largely on a news release. But we do know that the segment did not include any perspectives from independent experts unaffiliated with the experiments.