Simplified method of item analysis as compared with traditional method

 

Acharya N A^1*, Dixit J V²

 

¹Assistant Professor, ²Professor and HOD, Department of Community Medicine, Government Medical College, Latur, Maharashtra, INDIA.

Email: drnamratathakur@gmail.com

 

Abstract               Background: Item analysis is an important component in the assessment of students and the examination program. The analysis is done by calculation of difficulty index and discrimination index. The conventional method which divides the students in 3 groups is tedious and has some technical difficulties. To overcome them, we have attempted to suggest simplified method. Objective: To assess the simplified method of item analysis against conventional method. Material and Methods: The present cross sectional study was conducted at Government Medical College, Latur in November 2015. Conventional method divides the students in 3 groups but we have divided them in only 2 groups. Item analysis was done by both conventional and simplified method for 10 MCQs which were answered by students of 1^st semester. The standard error of difference between proportions was calculated to test the difference in two methods. Results: The difficulty index and discrimination index calculated by conventional and new simplified method were statistically comparable. But the advantage of new method is that, it is easier, simpler and has few technical difficulties. Conclusion: As new method for item analysis is shorter, simpler and statistically comparable with conventional method, it can be a good substitute for traditional method.

Key Words: Item analysis, evaluation, medical education.

 

 

INTRODUCTION

Item analysis is a process which examines student responses to individual test items (questions) in order to assess the quality of those items and of the test as a whole.¹ It is an important component in the assessment of students and the examination program. The two indices that are calculated in item analysis are difficulty index and discrimination index. The method currently being adopted considers only high performers (upper 1/3^rd) and low performers (lower 1/3^rd) for calculating these indices. The “average performers” (middle1/3^rd) are omitted from the analysis.^2,3 It becomes a tedious job to divide students in three groups and then omit the middle group. Also, it may not be always possible to divide the students in three groups as many a times student enrollment in a batch is 50/100/150/200. Thus the method has technical difficulties. There is a need for a simplified method. We have attempted one such method in the present study.

 

MATERIAL AND METHODS

The present cross sectional study was conducted at Government Medical College, Latur in November 2015. The students of 1^st semester were given 10 MCQs on the topics taught in October. Duration of exam was 15 minutes. 60 students attended the test and returned the answer sheets. There was no negative marking while scoring. Item analysis was done for all 10 MCQs using both conventional method and simplified method.

Conventional method: The students were arranged in ascending order of merit and middle 1/3^rd students (20 students) were omitted from analysis. The difficulty index (P) and discrimination index (D) were calculated for all 10 MCQs by the formula as shown below.^2,3

P= (H+L)/T x100

D= (H-L)/T X 2

Where, H- high performers (upper 1/3^rd)

L- low performers(lower 1/3^rd)

T- total students. (40)

New simplified method: For using the simplified method, the students were divided in only 2 groups (upper ½ and lower ½) and the difficulty index (P) and discrimination index (D) were calculated for all 10 MCQs by the formula as shown below.

P= C / T x100

D= (H-L)/T X 2

Where,

C- Total number of students who have provided correct answer. (as we have divided students in only two groups H+L = Total number of students who have provided correct answer)

H- high performers (upper ½ )

L- low performers(lower ½ )

T- total students. (60)

Here, difficulty index can be calculated without dividing the papers in groups, as it is simply the percentage of students who have attempted correct answer. Thus it becomes much simpler method. The discrimination index was calculated by using similar formula as used in conventional method but the students were divided in two groups (upper ½ and lower ½). The data obtained by both the methods was entered in Microsoft excel and analyzed. The standard error of difference between proportions was calculated to test the difference in two methods.

 

Table 1: Comparison of difficulty index by conventional method and new simplified method

*Difference was statistically not significant

               

Table 2: Comparison of discrimination index by conventional method and new simplified method

*the difference was statistically not significant

DISCUSSION

Binet and Simon (1916) were among the first to systematically validate test items⁴ They had calculated the “item difficulty index” which was defined as the percentage of persons passing an item and denoted by ‘p’. This is one of the statistics used in classical item analysis. The difficulty index was calculated by other authors ^5,6 using same formula. i.e. (number of students answering an item correctly / number of students tested) X 100. Item discrimination compares the number of high scorers and low scorers who answer an item correctly. The percentage of individuals included in the highest and lowest groups can vary. Also, some variation in formula used for calculating discrimination index is observed. Kelly et al⁷ divided the students in three groups and included upper 27% and lower 27% in analysis. They have used number of people in the larger of the two groups as denominator. Nunnally⁸ suggested 25 percent, while SPSS (1999) uses the highest and lowest one-third. Gajjar et al⁹, Shete et al¹⁰ and Chavada et al¹¹ have also used upper and lower thirds to calculate item discrimination index. Sarin et al⁹ have done item analysis of MCQs and divided the students in 2 groups as done in present study. Thus, variability is observed in selecting the higher and lower groups in various studies. So, in present study we have attempted to compare the effect of selection of groups on item analysis. We have not found any statistical difference in conventional method and new simplified method.

 

CONCLUSION AND RECOMMENDATION

The new method for item analysis is shorter and simpler. It is statistically comparable with conventional method. Thus it can be a good substitute for traditional method.

 

 

REFERENCES

• https://www.washington.edu/oea/services/scanning_scoring/scoring/item_analysis.html last assessed on 10th February 11, 2016.
• Ananthkrishnan N. The item analysis. Medical Education Principles and Practice. 2nd ed. Pondicherry: JIPMER; 2000. p. 131-7.
• Tejinder S, Piyush G, Daljit S. Principles of Medical Education. 3rd ed. India: Jaypee Brothers, Medical Publishers Pvt. Limited; 2009. p. 70-7.
• A comparison of three types of item analysis in test development using classical and latent trait methods by Iris G. Benson. A dissertation presented to the Graduate Council of the University of Florida in partial fulfillment of the requirements for the degree of doctor of philosophy. University of Florida, 1977.
• Thorndike, R. M., Cunningham, G. K., Thorndike, R. L., and Hagen, E. P. (1991). Measurement and evaluation in psychology and education (5th Ed.). New York: MacMillan.
• Crocker, L., and Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart and Winston.
• Kelley, T. I. (1939). The selection of upper and lower gruops for the validation of test items, Journal of Educational psychology, 30,(1), 17-24.
• Nunnally, J. C. (1972). Educational measurement and evaluation (2nd Ed.). New York: McGraw-Hill.
• Gajjar S, Sharma R, Kumar P, Rana M Item and Test Analysis to Identify Quality Multiple Choice Questions (MCQs) from an Assessment of Medical Students of Ahmedabad, Gujarat Indian Journal of Community Medicine 2014, 39(1):17-20.
• Shete A, Kausar A, Lakhkar K, Khan S. Item analysis: An evaluation of multiple choice questions in Physiology examination. J Contemp Med Edu 2015;3(3): 106-9.
• Chavda P, Misra S, Duttaroy B. Item Analysis of Multiple Choice Questions based Undergraduate Assessment in Community Medicine. South East Asian Journal of Medical Education 2015; 9(1): 66-8.
• Sarin Y, Khurana M, Natu M, Thomas A, Singh T Item analysis of published MCQs. Indian Paediatrics 1998; 35:1103-5.