Performance of GPT‐4o in the management of toxicological exposures: A comparative analysis with emergency medicine residents

Abstract Background Timely decision‐making is critical for managing toxicological exposures; however, many physicians lack toxicology‐specific experience and resources. Objectives This study aimed to assess whether GPT‐4o, an artificial intelligence (AI)‐based decision support system, could provide emergency medicine residents with information for managing toxicology cases and to analyze its potential effectiveness in guiding treatment decisions compared to toxicologists. Methods We conducted a prospective observational study with 30 emergency medicine residents (16 junior residents [JRs] and 14 senior residents [SRs]) using GPT‐4o. Each resident was assigned 30 clinical scenarios derived from real‐life toxicological exposure cases. GPT‐4o responded to the same 30 scenarios assigned to each resident. The responses from the residents and GPT‐4o were compared to gold standard (GS) responses determined by a medical toxicologist. Cohen's kappa coefficient was used to evaluate the agreement between each group's responses and the GS. Results GPT‐4o and SRs showed a similar and good agreement with the GS for recommending antidotal treatment (GPT‐4o: κ = 0.710, p