ITEM ANALYSIS OF MULTIPLE-CHOICE QUESTIONS IN AN UNDERGRADUATE SURGERY COURSE-AN ASSESSMENT OF AN ASSESSMENT TOOL

Lamees Sulaiman Al Sulaim; Sajad Ahmad Salati

doi:10.5937/sanamed0-50691

Lamees Sulaiman Al Sulaim Qassim University, College of Medicine, Department of Surgery, Unaizah, Kingdom of Saudi Arabia
Sajad Ahmad Salati Qassim University, College of Medicine, Department of Surgery, Unaizah, Kingdom of Saudi Arabia

DOI: https://doi.org/10.5937/sanamed0-50691

Keywords: Assessment;, Item analysis, Multiple-choice questions, Facility index, Discriminative index, Distractor efficiency, Non-functional distractor, Distractor analysis.

Abstract

Introduction: In the field of medical education, multiple-choice questions (MCQs) represent the most commonly utilized method of assessment. It is necessary to analyze the assessment results through item analysis to ensure the quality is appropriate. This study evaluated the quality of the MCQs utilized for summative evaluation of the students in the General Surgery Course conducted in the year 2023-24, at the College of Medicine (Unaizah), Qassim University, Saudi Arabia.

Methods: Using a number of established parameters for item analysis, the study evaluated the multiple-choice questions for difficulty, discrimination power, and quality of distractors.

Results: The quality of the questions varied. The means of the facility index, discrimination index, discriminative efficiency, and distractor efficiency were, in order, 76.31%, 0.28, -0.7743, and 32%.

Conclusion: Item analysis is a crucial technique for evaluating the quality of MCQs. There were multiple defects in the MCQs used in summative assessments, revealing the scope for further improvement in future courses. It is important to plan faculty development events often to impart knowledge and skills related to creating MCQs that are valid, reliable, and of high quality.

References

Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676-82. doi: 10.3109/0142159X.2010.500704.

Rae MG, Abdulla MH. An investigation of preclinical medical students' preference for summative or formative assessment for physiology learning. Adv Physiol Educ. 2023 ;47(3):383-92. doi: 10.1152/advan.00013.2023.

Arjoon JA, Xu X, Lewis JE. Understanding the state of the art for measurement in chemistry education research: Examining the psychometric evidence. J Chem Educ. 2013;90(5):536-45. doi: 10.1021/ed3002013.

Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 2004;38(9):974-9. doi: 10.1111/j.1365-2929.2004.01916. x.

Kar SS, Lakshminarayanan S, Mahalakshmy T. Basic principles of constructing multiple choice questions. IJCFM. 2015; 1(2):65-9. doi:10.4103/2395-2113.251640.

Towns M.H. Guide to developing high-quality, reliable, and valid multiple-choice assessments. J Chem Educ. 2014; 91(9):1426-31. doi:10.1021/ed500076x.

Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: A descriptive analysis. BMC Med Educ. 2009; 9:40. doi: 10.1186/1472-6920-9-40.

Hingorjo MR, Jaleel F. Analysis of one-best MCQs: the difficulty index, discrimination index and distractor efficiency. J Pak Med Assoc. 2012;62(2):142-7.

Odukoya JA, Adekeye O, Igbinoba AO, Afolabi A. Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university. Qual Quant. 2018;52(3):983-97. doi: 10.1007/s11135-017-0499-2.

Frey BB, Petersen S, Edwards L M, Pedrotti J T, Peyton V. Item-writing rules: Collective wisdom. Teaching and Teacher Education, 2005;21:357-64. doi: 10.1016/j.tate.2005.01.008

Mc Coubrie P. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 2004;26(8):709-12. doi: 10.1080/01421590400013495.

Downing SM. The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Adv Health Sci Educ Theory Pract. 2005;10(2):133-43. doi: 10.1007/s10459-004-4019-5.

Abdulghani HM, Ahmad F, Irshad M, Khalil MS, Al-Shaikh GK, Syed S, et al. Faculty development programs improve the quality of Multiple-Choice Questions items' writing. Sci Rep. 2015;5:9556. doi: 10.1038/srep09556.

Lai H, Gierl MJ, Touchie C, Pugh D, Boulais AP, De Champlain A. Using Automatic item generation to improve the quality of MCQ distractors. Teach Learn Med. 2016;28(2):166-73. doi: 10.1080/10401334.2016.1146608.

Chauhan P, Chauhan GR, Chauhan BR, Vaza JV, Rathod SP. Relationship between difficulty index and distracter effectiveness in single best-answer stem type multiple choice questions. Int J Anatomy Res. 2015; 3(4): 1607-10. doi: 10.16965/ijar.2015.299

Patel R M. Use of Item analysis to improve quality of Multiple-Choice Questions in II MBBS. J EducTechnol Health Sci. 2017;4(1):22-9.

Mahjabeen W, Alam S, Hassan U, Zafar T, Butt R, Konain S, et al. Difficulty index, discrimination index and distractor efficiency in multiple choice questions. Ann Pak Inst Med Sci. 2017; 13(4):310-5. doi: 10.48036/apims.v13i4.9.

Rao C, Prasad HK, Sajitha K, Permi H and Shetty J. Item analysis of Multiple- Choice Questions: Assessing an assessment tool in medical students. Int J Educational Psychol Res. 2017; 2(4):201-4. doi: 10.4103/2395-2296.189670.

Gajjar S, Sharma R, Kumar P, Rana M. Item and test analysis to identify quality Multiple Choice Questions (MCQs) from an assessment of medical students of Ahmedabad, Gujarat. Indian J Community Med. 2014;39(1):17-20. doi: 10.4103/0970-0218.126347.

Lama C P, Kharbuja R, Karki D, Dhungel S. Study on item analysis of Multiple-Choice Questions amongst the undergraduate dental students. Nepal Med Coll J. 2023; 25 (4): 301-5. doi: 10.3126/nmcj.v25i4.60876.

Bhat SK, Prasad KH. Item analysis and optimizing multiple‑choice questions for a viable question bank in ophthalmology: A cross‑sectional study. Indian J Ophthalmol. 2021; 69(2):343-6. doi: 10.4103/ijo.IJO_1610_20.

Baste VS. Item analysis of MCQs in physiology and its correlation with faculty’s perception of difficulty level of MCQs. Natl J Physiol Pharm Pharmacol. 2023;13 (7): 1444-8. doi: 10.5455/njppp.2023.13.12601202220122022.

AlFaris E, Naeem N, Irfan F, Qureshi R, Saad H, Al Sadhan R, et al. A one-day dental faculty workshop in writing Multiple-Choice Questions: An impact evaluation. J Dent Educ. 2015 ;79(11):1305-13.

Ebrahimi S, Kojuri J. Assessing the impact of faculty development fellowship in Shiraz University of Medical Sciences. Arch Iran Med. 2012 ;15(2):79-81.

Elliot DL, Skeff KM, Stratos GA. How do you get to the improvement of teaching? A longitudinal faculty development program for medical educators. Teach Learn Med. 1999; 11:52-7. doi: 10.1207/S15328015TLM1101_12.