(The Hague) In the emergency room, the artificial intelligence dialog ChatGPT performed diagnoses at least as well as doctors and in some cases outperformed them, Dutch researchers found, saying AI could “revolutionize the medical field “.
The authors of the study published Wednesday, however, stressed that the days of emergency doctors were not yet numbered, the dialoguer being potentially capable of speeding up the diagnosis, but not of replacing the judgment and experience of a human.
Thirty cases treated in an emergency department in the Netherlands in 2022 were reviewed by populating ChatGPT based on patient history, laboratory tests and physician observations, asking the chatter to suggest five possible diagnoses.
In 87% of cases, the correct diagnosis was found in the practitioner list, compared to 97% for ChatGPT version 3.5.
The dialoguer “was able to make medical diagnoses much like a human doctor would have done,” summarized Hidde ten Berg, from the emergency department of Jeroen Bosch Hospital in the south of the Netherlands.
Study co-author Steef Kurstjens stressed that it did not conclude that computers could one day run emergency rooms, but that AI could play a vital role in helping doctors under pressure.
The dialoguer “can help make a diagnosis and can perhaps suggest ideas that the doctor had not thought of,” he told AFP.
Such tools are not designed as medical devices, he noted, however, also sharing concerns about the confidentiality of sensitive medical data in a dialog.
And, as in other areas, ChatGPT has encountered some limitations.
His reasoning was “sometimes medically implausible or inconsistent, which can lead to misinformation or incorrect diagnosis, with significant implications,” the study notes.
The scientists also admit to some shortcomings in their research, such as the small sample size.
Additionally, only relatively simple cases were reviewed, with patients presenting with only one main complaint. The effectiveness of the dialoguer in complex cases is unclear.
Sometimes ChatGPT did not provide the correct diagnosis in all five possibilities, Kurstjens said, particularly in the case of an abdominal aortic aneurysm, a potentially life-threatening complication, with the aorta swelling.
But, consolation for ChatGPT, in this case the doctor was also wrong.
The report further notes medical “blunders” made by the caller, for example diagnosing anemia (low hemoglobin in the blood) in a patient with normal hemoglobin.
The results of the study, published in the specialist journal Annals of Emergency Medicine, will be presented at the 2023 European Congress of Emergency Medicine (EUSEM) in Barcelona.