Pdf on jul 19, 2018, alireza ahmadi and others published differential item functioning find, read and cite all the research you need on researchgate. Differential item functioning for minority examinees on the. This study used confirmatory factor analysis to examine possible differential item functioning dif across english and spanish versions of the ptsd checklistcivilian version pclc. Recommendations for conducting differential item functioning. As demonstration, a dataset from a 40item math test in a statewide testing program is analyzed by two widely. Item response theory detects differential item functioning. In brief, differential item functioning dif occurs when groups such as defined by gender, ethnicity, age, or education have different probabilities of endorsing a given item on a multi item scale after controlling for overall scale scores.
A general framework and an r package for the detection of. Understanding differential item functioning and item bias. An item is said to possess differential item functioning when the area index is greater than a critical value of 0. Recent advances in analysis of differential item functioning in health research using the rasch model curt hagquist1 and david andrich2 abstract background. Differential item functioning in the unified dyskinesia. Item response theory detects differential item functioning between healthy and ill children in qol measures. To demonstrate the value of item response theory irt and differential item functioning dif methods in examining a healthrelated quality of life hrqol measure in children and adolescents.
Potential sources of differential item functioning in the adaptation of tests paula elosua and alicia lopezjauregui department of psychology university of the basque country, spain this report shows a classification of differential item functioning dif sources that have an effect on the adaptation of tests. Determination of a differential item functioning procedure. This article provides a brief primer overview of differential item functioning dif analysis. Differential item functioning dif is a statistical characteristic of an item that shows the extent to which the item might be measuring different abilities for members of separate subgroups. Pdf an introduction to differential item functioning. Differential item functioning columbia university mailman. Detecting differential item functioning using logistic. Detecting and analyzing differential item functioning. Logistic regression modeling as a unitary framework for binary and likerttype ordinal item scores. Differential item functioning in pisa due to mode effects. Assessment of differential item functioning in the. A statistic for testing the hypothesis of no dif is developed. Goh, and lee ong kim national institute of education, nanyang technological university differential item functioning dif analysis is a way of determining whether test items function.
Detecting and analyzing differential item functioning in an essay test using the partial credit model steven ferrara leslie walkerbartnick maryland state department of education april ii, 1990 in recent years a great deal of measurement research has focused on various procedures for detecting differential. We analyzed 95 cognitive reading items, administered to students in 29 european countries. The aim of this research is to compare the result of the differential item functioning dif determining with hierarchical generalized linear model hglm technique and the results of the dif determining with logistic regression lr and item response theorylikelihood ratio irtlr techniques on the test items. Thayer journal of educational and behavioral statistics 1996 21. Differential item functioning dif has been increasingly applied in fairness studies in psychometric circles. Perhaps the item is tapping a secondary factor or factors overandabove the one of interest.
Several methods have been proposed in recent decades for identifying items that function differently between two or more groups of examinees. Bechger and maris proposed using a differential item pair functioning dif test, which focuses on comparing item pairs instead of seeing dif as an item property. An investigation of differential item functioning in the melab listening test vahid aryadoust, christine c. The use of the term bias refers to various contexts, both statistical and social. First, performance on the item is influenced by sources other than differences on the construct of interest that are deemed to be detrimental to one group. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods, an r.
An examination of the improved wald test for differential. Burton, the effect of item screening on test scores and test characteristics. Haladyna and downing 2004 identified potential sources of systematic errors associated with constructirrelevant variance, that included factors relating to test development. Differential item functioning for minority examinees on. If differential item functioning exists, interpretation of item scores must consider secondary influences on dyskinesia ratings.
This illustration uses data from 5,429 children using the. The difference with traditional procedures is that dif is defined in terms of the relative difficulties of pairs of itemswhich are identified from the observationsand not in terms. Starting from a framework for classifying dif detection methods and from a comparative overview of the most traditional methods. A logistic regression model for characterizing differential item functioning dif between two groups is presented. Differential item functioning dif analysis is a key component in the evaluation of the fairness and validity of educational tests. International comparison based on questionnaire scores. Dif studies are generally implemented to see how the performances of two groups compare on a single item in an assessment though studies can be done with more than two groups. An item is labeled as having dif when people with the. See the following manual entries for details about the individual dif tests, including syntax and worked. An item is considered to be biased if the item unfairly favors one group over another. X fits an item response model when x are item scores e. Assessment of differential item functioning in the experiences of discrimination index. Another area of item performance that is often used for item retention decisions is item bias, commonly referred to as differential item functioning dif.
Average item scores for subgroups having the same overall score on the test are compared to determine whether the item is measuring in essentially the. Spanish and english versions of the ptsd checklist. Differential item functioning sage research methods. Using unified dyskinesia rating scale translation databases n 3,2, we tested uniform and nonuniform differential item functioning.
Differential item functioning analysis of the copsoq across. Dif detection generally focus on the identification of uniform and nonuniform dif, where. Differential item functioning dif occurs when items that are intended to measure a latent trait are unfair, favoring one group of individuals over another. Dif analysis investigates a differential characteristic of a test item between subpopulations of examinees and is useful in detecting possibly biased items toward a particular subpopulation. Evaluating the magnitude of differential item functioning. Table 30 supports the investigation of item bias, differential item functioning dif, i. An introduction to differential item functioning analysis eric. This issue, known as test bias, has been the subject of a great deal of recent research, and a technique called differential item functioning dif analysis has become the new standard in psychometric bias analysis. Dorans, evaluating hypotheses about differential item functioning. The coronary artery risk development in young adults cardia study timothy j.
Judicious application of this methodology by the researchers, however, requires an understanding of the technical complexities involved. Cunningham, centers for disease control and prevention, 4770 buford highway ne, mailstop k22, atlanta, ga 30341 email. The purpose of dif analyses is to detect response differences of items in questionnaires, rating scales, or tests across different. Differential item functioning dif is an important issue of interest in psychometrics and educational measurement. Paper 29002015 multiple ways to detect differential item. The purpose of the present analysis is to use differential item functioning dif to identify differences in the performance of native and immigrant students in pisa 2009 that can be directly related to their responses to particular items. The rows in each group refer to the levels from lower to higher, with the fourth row indicating the sum of each ability level. Evaluating the magnitude of differential item functioning in polytomous items rebecca zwick and dorothy t. This simulation study examines itemlevel differential item functioning dif in the context of international largescale assessment ilsa using a generalized logistic regression approach.
The study examined differential item functioning in social studies multiple choice questions in basic education certificate examination. Assessment developers design and construct questionnaires or tests including sets of items that measure, for example, cognition, personality traits, or political views. Differential item functioning dif has been widely used in healthcare, business management, and educational measurement. Multilevel generalized mantelhaenszel for differential. Differential item functioning dif refers to group differences in performance on a test item that cannot be explained by group differences in the construct targeted. Data were derived from two assessments of hispanics n s 304, 2, who were hospitalized with physical injuries. Differential item functioning analysis of the copsoq. Judicious application of this methodology by the researchers, however, requires an. Differential item functioning dif in psychometric tests has long been recognized as a potential source of bias in person measurement. Sep 01, 2017 this is the webpage for the handbook on differential item functioning. This difference in the item score, above and beyond group differences on the construct, is referred to as differential item functioning dif.
Measurement invariance and differential item functioning. If the factor bringing about such a difference is not part of the construct of focus in the test, then the test would be biased. Differential item functioning dif, as an assessment tool, has been widely used in quantitative psychology, educational measurement, business management, and insurance and healthcare industries. More specifically, an item is considered to be biased if two conditions are met.
Detecting and analyzing differential item functioning in an essay test using the partial credit model steven ferrara leslie walkerbartnick maryland state department of education paper presented at the annual meeting of the national council on measurement in education, april 19, 1990, boston. Detecting and analyzing differential item functioning steven. Current issues 951 people who answered the item correctly at the ability level m, and the proportion of people who answered the item correctly at the ability level m, respectively. Recently, it has been applied to mental healthquality of life measurement. Three statistical testing procedures in logistic regression. Differential item functioning dif is a direct threat to the mi of test items and occurs when item parameters differ across equal ability groups, resulting in the differential likelihood of a particular e. Many dif methods have been investigated and developed, including the. A handbook on the theory and methods of differential item functioning dif. Lewis, a note on the value of including the studied item in the test score when analyzing test items for dif. Spanish and english versions of the ptsd checklist civilian. Rasch analysis with a focus on differential item functioning dif is increasingly used for examination of psychometric properties of health outcome measures. Five research questions and five hypotheses guided the study. Examining differential item functioning in reading. Recent advances in analysis of differential item functioning.