SNIP
Two artificial intelligence (AI) programs — including ChatGPT — have passed the U.S. Medical Licensing Examination (USMLE), according to two recent papers.
The papers highlighted different approaches to using large language models to take the USMLE, which is comprised of three exams: Step 1, Step 2 CK, and Step 3.
…
The first paper, published on medRxiv in December, investigated ChatGPT’s performance on the USMLE without any special training or reinforcement prior to the exams. According to Victor Tseng, MD, of Ansible Health in Mountain View, California, and colleagues, the results showed “new and surprising evidence” that this AI tool was up to the challenge.
Tseng and team noted that ChatGPT was able to perform at >50% accuracy across all of the exams, and even achieved 60% in most of their analyses. While the USMLE passing threshold does vary between years, the authors said that passing is approximately 60% most years.
“ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement,” they wrote, noting that the tool was able to demonstrate “a high level of concordance and insight in its explanations.”
Source:
www.medpagetoday.com/special-reports/exclusives/102705
Link to the journal article:
www.medrxiv.org/content/10.1101/2022.12.19.22283643v1