Recap of small study on schizophrenia diagnostic device offers false hope

Does the news release adequately discuss the costs of the intervention?

Not Satisfactory

There was no discussion of the cost of this device (or the cost of a test using the device). One news release put the cost at “less than $15,000.”

Does the news release adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The news release stated the results “show the device accurately indicated reduced electrical activity in the retina in multiple cell layers in the participants who had schizophrenia, including in cell types that had not been studied before in this disorder.”

Presumably this meant that all of the patients with schizophrenia were identified, but there are no numbers. Also, the news release didn’t explain whether the device accurately ruled out schizophrenia in those who didn’t have it.

The news release and scientific article both referred to group (averaged) data, which is distinct from individual diagnosis.

Does the news release adequately explain/quantify the harms of the intervention?

Not Satisfactory

What’s known about the safety of this device? And what about the chances of false positives or false negatives? The news release didn’t say. It only quoted a researcher stating the test was performed to be “as noninvasive and quick as possible.”

Does the news release seem to grasp the quality of the evidence?

Not Satisfactory

There was no mention of limitations of this study, such as the small number of participants. It also wasn’t clear whether the study was blinded; presumably at least the study participants if not the researchers taking the measurements knew their status.

A study to show the diagnostic usefulness of the device would require a much larger sample of mostly younger people, positive and negative predictive values, and involve patients with other conditions such as mood and drug abuse disorders.

Does the news release identify funding sources & disclose conflicts of interest?

Not Satisfactory

The news release did not mention who funded this study, and it which wasn’t mentioned in the study itself, as far as we could tell. The release also didn’t mention that one of the authors is employed by LKC Technologies, which makes these devices.

Does the news release compare the new approach with existing alternatives?

Not Satisfactory

There was no discussion of how schizophrenia is diagnosed currently, which involves a checklist of symptoms such as delusions, incoherent speech, hallucinations, hearing voices, cognitive problems, or a flattening of emotions that hang on for months. How accurate is that method, and how much does it cost?

Does the news release establish the true novelty of the approach?

Not Satisfactory

The news release said was this is the first time a portable device was used for these tests.

The news release also notes that, “Looking at biomarkers in the eye as a way to understand psychiatric disorder is a new field of study,” but the journal article cites studies as far back as 1999. So not so new.

Does the news release include unjustifiable, sensational language, including in the quotes of researchers?

Not Satisfactory

The lead sentence oversold this study, indicating that it showed how a common device “may hold the key to faster diagnosis of schizophrenia, predicting relapse and symptom severity and assessing treatment effectiveness.”

The news release headline also over-reaches. A better headline might have been the sub-title: “Researchers explore eye function in schizophrenia as a window into the brain.”

But further down the news release quoted a researcher stating that the study “should help generate further research” into developing a test clinicians can use to diagnose, treat and monitor people with schizophrenia.

And at the end a quote — from lead author Docia Demmin — tempered the assertion by stating “it’s too soon to call this a diagnostic tool.”