When people feel unwell, researching symptoms online is one of the easiest and quickest ways to find information that might be able to help explain the problem.
Unfortunately, getting a DIY diagnosis via the internet is likely to be wrong more often that not, according to new research that might make you think twice about Googling your symptoms next time you get sick.
A study analysing the accuracy of online symptom-checking websites and apps reveals that the quality of their diagnostic advice varies considerably, and on average they only list the correct diagnosis first about one-third (36 percent) of the time.
“While it may be tempting to use these tools to find out what may be causing your symptoms, most of the time they are unreliable at best and can be dangerous at worst,” says study first author and masters student Michella Hill from Edith Cowan University (ECU) in Australia.
“The reality is these websites and apps should be viewed very cautiously as they do not look at the whole picture.”
To investigate online symptom checkers (SCs) providing medical diagnosis or triage advice, Hill and fellow researchers identified 36 popular and free SCs available through websites or via apps for iPhones and Android smartphones.
With these SCs in hand, they then tested them against 48 medical condition descriptions, based on the symptoms patients tend to present when experiencing various illnesses. Some of these patient vignettes were drawn from previous research and healthcare professional training materials, while others were new symptom‐based scenarios reflecting a number of Australia‐specific illnesses.
Ultimately, the SCs delivered mixed results. In the 27 services that produced diagnostic information based on the symptoms provided, the SCs listed the correct diagnosis first in just 36 percent of tests – meaning roughly two-thirds of the time, their best guess for what might ail you is wrong.
Interestingly, a previous study published five years ago by a team of researchers at Harvard University found almost exactly the same figure in their own experiment testing this, in which 23 SCs suggested the correct diagnosis first in just 34 percent of cases.
That similarity could have something to do with the fact that both studies shared some of the same medical condition vignettes.
But given the new study also brought in new conditions – and didn’t necessarily test them against the same SCs – it also suggests something else: on the whole, in the last half-decade, these online symptom checkers haven’t really become any more reliable, at least in terms of pinpointing the single, most probable illness a patient might have.
Not that they’re totally useless, either. In the new study, Hill’s team found SCs listed the correct diagnosis in the top ten results 58 percent of the time. Even so, clearly there’s still plenty of refinement to be done here.
One area where SCs fared a little better was in triage advice – instances where, based on the symptoms provided, the apps and websites would recommend people seek medical care. Even then, though, the results were a bit of a mixed bag.
“The advice for seeking medical attention for emergency and urgent care cases was appropriate around 60 percent of the time, but for non-emergencies that dropped to 30 to 40 percent,” Hill says.
“Generally the triage advice erred on the side of caution, which in some ways is good, but can lead to people going to an emergency department when they really don’t need to.”
According to the researchers, SCs that use artificial intelligence algorithms and base their advice on a spread of demographic information are more reliable than the rest. Unfortunately, not all SCs have such sophisticated back-ends, and even if they do, differences in AI algorithms and programming can mean you get inconsistent results.
On the plus side, at least these services are uniformly upfront about one aspect of their limitations.
“Each included SC warned that their service was not a substitute for consulting a physician,” the authors explain.
“Diagnosis is not a single assessment, but rather a process requiring knowledge, experience, clinical examination and testing, and the passage of time, impossible to replicate in a single online interaction.”
Despite the shortcomings, the researchers say SCs are useful, but only if people use them as an educational resource in conjunction with consultations from professional health care services.
Whatever advice these algorithms generate for us, it’s important we don’t mistake that for an actual medical diagnosis coming from a physician, because these digital repositories are many things, but they’re definitely not doctors.
“They don’t know your medical history or other symptoms,” Hill says. “For people who lack health knowledge, they may think the advice they’re given is accurate or that their condition is not serious when it may be.”
The findings are reported in The Medical Journal of Australia.
