Item request has been placed! ×
Item request cannot be made. ×
loading  Processing Request

Keeping humans in the loop efficiently by generating question templates instead of questions using AI: Validity evidence on Hybrid AIG.

Item request has been placed! ×
Item request cannot be made. ×
loading   Processing Request
اقرأ على الانترنت اقرأ أكثر حفظ في قائمتي
  • معلومة اضافية
    • الموضوع:
      GENERATIVE artificial intelligence; MEDICAL logic; MEDICAL education; PROFESSIONS; PSYCHOMETRICS; AUTOMATION; NATIONAL competency-based educational tests; SOFTWARE architecture; TEST design; COMPUTER assisted testing (Education); PROFESSIONAL competence; CHATBOTS
    • نبذة مختصرة :
      Background: Manually creating multiple-choice questions (MCQ) is inefficient. Automatic item generation (AIG) offers a scalable solution, with two main approaches: template-based and non-template-based (AI-driven). Template-based AIG ensures accuracy but requires significant expert input to develop templates. In contrast, AI-driven AIG can generate questions quickly but with inaccuracies. The Hybrid AIG combines the strengths of both methods. However, neither have MCQs been generated using the Hybrid AIG approach nor has any validity evidence been provided. Methods: We generated MCQs using the Hybrid AIG approach and investigated the validity evidence of these questions by determining whether experts could identify the correct answers. We used a custom ChatGPT to develop an item template, which were then fed into Gazitor, a template-based AIG (non-AI) software. A panel of medical doctors identified the answers. Results: Of 105 decisions, 101 (96.2%) matched the software's correct answer. In all MCQs (100%), the experts reached a consensus on the correct answer. The evidence corresponds to the 'Relations to Other Variables' in Messick's validity framework. Conclusions: The Hybrid AIG approach can enhance the efficiency of MCQ generation while maintaining accuracy. It mitigates concerns about hallucinations while benefiting from AI. [ABSTRACT FROM AUTHOR]
    • نبذة مختصرة :
      Copyright of Medical Teacher is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)