Warning: early vaccine trial results don’t always stand test of time

“The drugmaker Moderna said on Monday that the first coronavirus vaccine to be tested in people appeared to be safe and able to stimulate an immune response against the virus.”

That’s what an early New York Times story reported yesterday.

Erick Turner, MD, reacted on Twitter:

Hey, didn’t you hear? The vaccine “appears to be safe”. That’s based on the results from eight people––eight!! Very reassuring! ?

Hard-nosed reporting––NOT!
FYI @garyschwitzer https://t.co/sUvQm8MAGN pic.twitter.com/E0qMNX0MEN

— Erick Turner (@eturnermd1) May 18, 2020

Story strengths & weaknesses

What follows is a brief collection of some strengths and weaknesses of other stories on the early vaccine results.

A Fox News story quoted a surgeon saying that it was “great news” because “I believe that to save American lives we need American medicine and American hands, not the Chinese government leading the way and that’s what we’re doing here in America so I’m very proud.”

NBC News called it “Breaking News” and used the word ‘encouraging’ five times in a short segment.  Today Show anchorwoman Hoda Kotb actually said, “I love it when I see the word ‘encouraging’ in a script.” (Can you imagine if she had said she was excited about encouraging early poll numbers for a political candidate?)

The Wall Street Journal put important caveats up high in the fourth paragraph. (If you’re not a subscriber, you won’t get to the fourth paragraph; it’s behind a paywall.)

The vaccine still has much to prove.  The results don’t show whether it actually protects people who are exposed to the new coronavirus, a key proof point.  Many vaccines fail to pass muster even after showing positive signs in early testing.

The WSJ went on to explain that:

The phase I study data came from among the 45 people ages 18 to 55 who received three different dose levels of the vaccine.  An additional 60 people over age 55 are being enrolled in the study.

That’s an interesting note, but it could have been presented much more clearly,  So the plan is to enroll 105 people in the phase I study.  45 people have been vaccinated.  But most stories only discussed results for 8 people.

CNN explained:

While the vaccine had promising results in the lab, it’s not known if it will protect people in the real world.

The New York Times did a better job explaining what was not real world in the early vaccine results reported.

The people vaccinated in Moderna’s Phase 1 study described on Monday were healthy volunteers ages 18 to 55. Their immune systems made antibodies that were then tested in infected cells in the lab, and were able to stop the virus from replicating — the key requirement for an effective vaccine.

So the meeting of antibodies and infected cells took place in the controlled setting of the lab – not in the bodies of sick people.

Still, that New York Times story was headlined, Moderna Coronavirus Vaccine Trial Shows Promising Early Results

Early results don’t always stand

That’s what a New York Times newsletter reminded readers:

In 2015, the French drug company Sanofi began selling the first vaccine for dengue. The drug had made it through multiple research trials — although some researchers believed Sanofi had ignored worrisome signs. Sure enough, as children in the Philippines began using it, some contracted an even worse form of dengue. Today, use of the vaccine is highly restricted.

In recent testimony, Dr. Anthony Fauci, the nation’s top infectious-disease expert, noted that a coronavirus vaccine could suffer from the same problem.

The larger point is that drugs that look good in small, initial studies often look less good when they’re tested in more people.

With such early trial results, you never hear much about safety or side effects.  You can’t learn much about safety in such small samples on a short-term basis.

None of this kept the manufacturer’s stock from soaring. None of it kept pundits from predicting demand and availability.

Meantime, Ed Silverman of STAT News reported on continuing “Where are the data?” questions – “Amid worldwide clamor for Covid-19 medicines and vaccines.”

Those are good questions.

As are the questions about former Moderna board member and pharma executive Moncef Slaoui – now the President’s newly appointed “vaccine czar” – first claiming he had no conflict of interest, and then announcing he would divest his equity holdings in Moderna.  But I’ll leave that to the political and financial journalists.

Addendum:

See this excellent analysis by Helen Branswell at STAT news – Vaccine experts say Moderna didn’t produce data critical to assessing Covid-19 vaccine

Also see an op-ed in the Washington Post by William Haseltine, former Harvard Medical School professor and founder of the university’s cancer and HIV/AIDS research departments. Moderna’s claim of favorable results in its vaccine trial is an example of ‘publication by press release’