Kevin Lomangino is the managing editor of HealthNewsReview.org. He tweets as @Klomangino.
For years, healthcare professionals have been warning us to be wary of Dr. Google.
But what about that new Dr. Bing who’s apparently getting ready to hang his shingle?
According to a report this week in the New York Times, Microsoft’s search engine has clinical powers that far outstrip those of Google or any other consumer technology application. Bing is not just going to answer your health questions, monitor your heart rate, or spit out lab test results; it’s going to give you an early heads up about your cancer simply by analyzing your search queries.
Here’s how the Times puts it:
Microsoft scientists have demonstrated that by analyzing large samples of search engine queries they may in some cases be able to identify internet users who are suffering from pancreatic cancer, even before they have received a diagnosis of the disease.
The story is based on a new study of Bing search queries published in the Journal of Oncology Practice. The researchers found individuals whose search patterns indicated they had been diagnosed with pancreatic cancer (for example, “why did I get cancer in pancreas,” and “I was told I have pancreatic cancer what to expect”). They then looked backward at these people’s searches before the diagnosis and identified patterns that were indicative of cancer. The results suggest that it’s possible to give these patients early warning about their diagnosis, and that this early warning “can increase the five-year survival rate of pancreatic patients to 5 to 7 percent, from just 3 percent,” according to the Times.
The Times coverage is notable for its utter lack of skepticism and its failure to challenge any of the concepts or claims made in the story. We don’t hear from any independent experts, even though the premise of the story is so extraordinary as to practically demand some outside scrutiny.
We hear exclusively from the Microsoft researchers, who consider themselves iconoclastic outsiders who are ready to shake up the medical establishment.
“I think the mainstream medical literature has been resistant to these kinds of studies and this kind of data,” Dr. Horvitz said. “We’re hoping that this stimulates quite a bit of interesting conversation.”
The conversation alluded to by Horvitz, one of the Microsoft study authors, could have and should have started with the Times’s coverage of the study. For me, the study raises important questions about the creeping use of technology — things like direct-to-consumer lab and genetic testing, wearable monitoring devices, and now even search engine queries — that have the potential to turn more people into patients without any proof that they’ll actually be helped.
I also reached out to Vinay Prasad, MD, MPH, a hematologist-oncologist and Assistant Professor of Medicine at the Oregon Health and Sciences University. He had scathing criticism for the story as well as the study that it was based on, starting with the premise that typing “I was just diagnosed with pancreatic cancer” into a search box (which is how the Microsoft researchers identified their cancer “cases”) means that someone does in fact have pancreatic cancer.
First, I would want to know the correlation between all of the search terms they used (which are not given) and the formal rate of diagnosis of pancreatic cancer — not given.
I would want to see of all the people who use these search terms, what percentage have no cancer, some cancer (and if so what type of cancer) — not given– also what stage do they have on diagnosis.
This is a prerequisite to even interest me.
Prasad was also highly skeptical of the authors’ claim that earlier diagnosis would make any meaningful difference to pancreatic cancer patients. He said that “having symptoms that alert you to pancreatic cancer probably nearly always means it is past the curable stage; thus the very premise of this paper seems dubious.” He continued:
There are several types of pancreatic cancer, but the most common, most well known and most feared in pancreatic adenocarcinoma. This is what struck down Patrick Swayze.
For a very small subset of pancreatic adenocarcinoma early detection (when the mass is very small) can lead to a surgery that cures these pts. For the vast majority of patients however, we cannot cure them, and our drugs and therapies may only extend their life very modestly– a handful of months at best.
Treating these incurable cancers with these drugs a little earlier is not likely to make much of a difference. In fact, we have failed and continue to fail patients with pancreatic adenocarcinoma as a field. No one will say we have been greatly successful. Even our celebrations have been over marginal improvements. Someone who is not in the field would look at what we are celebrating and ask if we are delusional.
I want to say the gap between this work and practice is as large as the grand canyon– but that doesn’t do it justice. The gap between this study and any usable information at all is earth to the next galaxy. It is cosmic distance. If the authors try to commercialize this project then they should start speaking with Elizabeth Holmes already about how to handle the fall out of a failed company.
Prasad’s reference to Holmes and her failed blood testing company, Theranos, is worth calling attention to. That was another Silicon Valley-style effort that made wild claims about shaking up the medical establishment with paradigm-shifting technology. And now the company is the subject of a criminal investigation related to those claims.
Microsoft’s effort is different but the issues it raises for journalists are the same — one of them being the need to vet extraordinary claims with care.
My perusal of other news stories related to this study turned up coverage that mostly followed the Times’s lead and didn’t include any outside perspective, including:
A broadcast radio discussion of the study on WNYC did include an independent panelist and some discussion of the ethics of mining people’s personal search data. But there was no scrutiny of the study itself or the claim that it would be worthwhile to mine people’s personal search queries to diagnose disease.