Scientific Validity Assessment and Optimization of AI-Generated A3/A4 Type Questions for the Chinese Medical Licensing Examination
Scientific Validity Assessment and Optimization of AI-Generated A3/A4 Type Questions for the Chinese Medical Licensing Examination: An Empirical Analysis Based on the "Answering-Generating" Closed Loop and Transformation of Competency Assessment
Guangdong Provincial People's Hospital
20 participants
Oct 1, 2025
OBSERVATIONAL
Conditions
Summary
This is a cross-sectional study that primarily employs quantitative analysis, supplemented by qualitative assessment. The research is conducted in two stages: Phase I consists of a model performance comparison experiment, and Phase II involves an item quality evaluation experiment. The entire study adheres to the principles of single-blinding, randomization, and standardization to ensure scientific rigor and reproducibility. The single-blind design is implemented during the "standardized testing" phase, where the system intersperses AI-generated items with those authored by human experts. Participants remain blinded to the source of each item (AI-generated vs. human-authored) throughout the testing and scoring processes, thereby ensuring the objectivity of the evaluation results.
Eligibility
Inclusion Criteria4
- Professional Status: Medical students currently enrolled in a Standardized Residency Training (SRT) program.
- Educational Background: Holders of a Bachelor of Medicine degree or higher, with foundational clinical knowledge.
- Informed Consent: Voluntarily participate in the study and provide written informed consent.
- Technical Competency: Proficient in using digital platforms to complete assessments and scoring.
Exclusion Criteria3
- Conflict of Interest: Individuals involved in the AI model training, prompt engineering, or the creation of the human-authored question bank for this study.
- Inability to Complete: Presence of visual/auditory impairments or severe illness that precludes completion of the assessment within the specified time.
- Investigator's Discretion: Any other condition that, in the opinion of the investigator, renders the participant unsuitable for the study.
Interested in this trial?
Get notified about updates and connect with the research team.
Locations(1)
View Full Details on ClinicalTrials.gov
For the most up-to-date information, visit the official listing.
NCT07505862