Bi-factor MIRT True-Score Equating for Testlet-Based Tests

다국어 초록 (Multilingual Abstract)

Lee and Lee (2014) proposed bi-factor multidimensional item response theory model (BFM) "observed-score" equating procedures. The main purposes of this study were to develop BFM "true-score" equating procedures, and to investigate applicability of the proposed procedures with actual data. Eight equating methods (including both true- and observed-score) based on dichotomous IRT (2-parameter logistic model: 2PL), polytomous IRT (graded response model: GRM), testlet response model (TRM), and BFM were compared with target equipercentile equating. Data for this study were from the Reading Comprehension test for a large-scale state assessment program, which consisted of several passages and corresponding groups of items. True- and observed-score equating methods based upon 2PL and BFM produced similar equating results. The GRM true- and observed-score equating methods provided equating results somewhat different from others, and more similar to the target equipercentile equating.

국문 초록 (Abstract)

Lee와 Lee(2014) 연구는 혼합형 검사에서의 bi-factor 다차원 문항반응모형을 적용한 IRT "관찰점수" 동등화 절차를 제안하였다. 이 연구는 Lee와 Lee(2014) 연구의 추수연구로 단위검사를 대상으로 bi-factor 다차원 문항반응모형을 적용한 IRT "진점수" 동등화 절차를 제안하고, 그 적용 가능성을 실제 자료를 이용하여 탐색하고자 하였다. 이를 위해 단위검사로 구성된 대규모 독해 검사 자료를 이용하여, 2모수 로지스틱 모형(2-parameter logistic model: 2PL), 등급반응모형(graded response model: GRM), 단위검사모형(testlet response model: TRM), bi-factor 모형(bi-factor model: BFM)을 적용하였으며, 관찰점수 및 진점수 동등화를 시행하여 총 8개의 동등화 방법을 적용하고, 산출된 결과를 동백분위 동등화 결과와 비교하였다. 연구 결과, 2PL과 BFM을 적용한 관찰점수 및 진점수 동등화는 유사한 결과를 산출하였으며, GRM을 적용한 관찰점수 및 진점수 동등화 방법은 다른 방법들과 다소 차이를 보이는 결과를 산출하였고, 비교 기준이 된 동백분위 동등화 결과와 보다 유사한 결과를 보였다.

목차 (Table of Contents)

• ABSTRACT
• Ⅰ. Introduction
• Ⅱ. Theoretical Backgrounds
• Ⅲ. Method
• Ⅳ. Results
• Ⅴ. Conclusions
• References
• <요약>

참고문헌 (Reference)

1 Ngudgratoke, S., "Vertical linking of test composed of testlets: A comparison between dichotomous models, polytomous models, and testlet response model" 2006

2 Min, K., "The impact of scale dilation on the quality of the linking of multidimensional item response theory calibrations" Michigan State University 2003

3 이규민, "Testlet Response Model for IRT True Score Equating" 한국교육평가학회 22 (22): 871-887, 2009

4 Kolen, M. J., "Test equating, scaling, and linking: Methods and practices" Springer 2014

5 Kolen, M. J., "Standard errors of Tucker equating" 9 : 209-223, 1981

6 Sireci, S. G., "On the reliability of testlet-based tests" 28 : 237-247, 1991

7 Brossman, B. G., "Observed score and true score equating procedures for multidimensional item response theory" 37 : 460-481, 2013

8 Yon, H., "Multidimensional item response theory (MIRT) approaches to vertical scaling" Michigan State University 2006

9 Reckase, M. D., "Multidimensional item response theory" Springer 2009

10 Hirsch, T., "Multidimensional equating" 26 : 337-349, 1989

11 Thompson, T., "Multidimensional IRT scale linking" 1997

12 Lee, W., "Mixed-format tests: Psychometric properties with a primary focus on equating (Volume 2)" Center for Advanced Studies in Measurement and Assessment, The University of Iowa 2012

13 Davey, T., "Linking multidimensional item calibration" 20 : 405-416, 1996

14 Lee, G., "Incorporating the testlet concept in test score analyses" 19 (19): 5-9, 2000

15 Cai, L., "IRTPRO: Flexible professional item response theory modeling for patient reported outcomes [Computer software]"

16 Petersen, N. S., "IRT versus conventional equating methods : A comparative study of scale stability" 8 : 137-156, 1983

17 Cook, L. L., "IRT equating: A flexible alternative to conventional methods for solving practica ltesting problems" 1981

18 Wainer, H., "How is reliability related to the quality of test scores? What is the effect of local dependence on reliability?" 15 : 22-29, 1996

19 Samejima, F., "Handbook of modern item response theory" Springer-Verlag 85-100, 1997

20 Cai, L., "Generalized full-information item bifactor analysis" 16 : 221-248, 2011

21 Gibbons, R. D., "Full-information item bifactor analysis" 57 : 423-436, 1992

22 Rijmen, F., "Formal relations and an empirical comparison among the bi-factor, the testlet, and a second-order multidimensional IRT model" 47 : 361-372, 2010

23 Lee, G., "Estimating reliability under a generalizability theory model for test scores composed of testlets" 12 : 237-255, 1999

24 Bock, R. D., "Estimating item parameters and latent ability when responses are scored in two or more nominal categories" 37 : 29-51, 1972

25 Lee, G., "Comparison of dichotomous and polytomous item response models in equating scores from tests composed of testlets" 25 : 357-372, 2001

26 Lord, F. M., "Comparison of IRT true-score and equipercentile observed-score “equatings.”" 8 : 452-461, 1984

27 Bishop, S. N., "Comparing vertical scales derived from dichotomous and polytomous IRT models for a test composed of testlets" 2002

28 Lee, G., "Bi-factor MIRT observed-score equating for mixed-format tests" 2014

29 DeMars, C. E., "Application of the bi-factor multidimensional item response theory model to testlet-based tests" 43 : 145-168, 2006

30 Wei, T., "An extension of IRT-based equating to the dichotomous testlet model" 2013

31 Li, Y. H., "An evaluation of the accuracy of multidimensional IRT linking" 24 : 115-138, 2000

32 Li, Y., "A test characteristic curve linking method for the testlet model" 29 (29): 340-356, 2005

33 Han, T., "A comparison among IRT true-and observed-score equating and traditional equipercentile equating" 10 : 105-121, 1997

34 Bradlow, E. T., "A Bayesian random effects model for testlets" 64 (64): 153-168, 1999