The Effect of Reducing the Number of Options on the Psychometric Properties of an MCQs test in a Medical School Hassan Bahaeldin Ahmmed* Assistant Professor, Department of Obstetrics and Gynecology, College of Medicine, King Khalid University, Guraiger, Abha, 62529, Saudi Arabia *Email id: bahasuikt@hotmail.com
Online published on 10 December, 2015. Abstract Background Single best answer multiple choice questions (MCQs) are one of the commonest assessment tools used in medical schools. Optimal numbers of options in single best answer MCQs is still a matter of debate that needs further evidence. Item analysis helps us to assess the quality of a particular test using test reliability, difficulty index, discrimination index and function of distracters as parameters for test quality. The aim of this study was to explore the effect of reducing the number of options of MCQs test in gynaecology from four to three on psychometric parameters of the same test. To the best of our knowledge, this is the first study to be conducted in a Saudi medical school with the mentioned objective for undergraduate medical students in a major clinical course. Methods Four-option MCQ test was conducted to a cohort of fifth-year medical students after completing their clinical course of obstetrics and gynaecology in a 6-year MBBS program incollege of medicine, King Khalid University, Saudi Arabia. The same test was modulated by removing the options with the least students’ responses. This reformed test was administered to another cohort of students from the same course. Item analysis of both tests was conducted. Paired t-test and chi-square tests were used to compare the psychometric parameters with P value set at 0.05 as significant. Results The three-option test showed a small but significant increase in test reliability compared to four options test (0.89 vs. 0.8, respectively). The mean item difficulty index of four-option test was 0.879 compared to 0.808 for the three-option test (P=0.0001). The mean item, discrimination index, was found to be 0.3111 and 0.2333 for the four-and three-option tests, respectively (P value =0.0001). Thirty five percent of the distracters in the three-option test were identified as non-functioning distracters (NFDs) compared to 49% in four-option test (P value= 0.001). The mean functioning distracter per item was found to be 1.27 for four and 1.53 for three-option tests (P value = 0.024). Conclusion The three-option test showed significant improvement in three of the four psychometric properties studied. This adds to evidence of using three options MCQs test in medical students’ assessment. Top Keywords MCQs, Reliability, Difficulty index, Discrimination index, Functioning distracters. Top |