공통문항 비동등집단 설계에서 구성형 문항의 IRT 관찰점수 동등화 방법 비교 = Comparison of IRT observed score equating methods for CR items under the common-item nonequivalent groups design

국문 초록 (Abstract)

이 연구는 공통문항 비동등집단 설계에서 구성형 문항의 IRT 관찰점수 동등화의 정확성을 알아보기 위한 모의실험 연구로, 다음 세 가지 조건을 사용하여 동등화의 정확성을 확인하였다. 첫째, 구성형 문항만을 공통문항으로 사용하여 일차원 IRT 모형을 적용하였다. 둘째, 구성형 문항과 외부(external) 선택형 문항을 공통문항으로 사용하여 일차원 IRT 모형을 적용하였다. 셋째, 구성형 문항과 외부 선택형 문항을 공통문항으로 사용하여 다차원 IRT 모형을 적용하였다. 연구 결과, 다음의 제한된 조건에서 공통문항에 구성형 문항만을 사용하는 것보다 선택형 문항을 함께 사용했을 때 동등화 결과가 더 정확하였다. 즉, 1) 다차원 IRT 모형을 적용하였을 때, 2) 표본 크기가 클 경우, 그리고 3) 선택형 문항에서 집단 간의 능력 차이가 작을 때 구성형 문항의 동등화의 정확성이 향상되었다. 또한, 일반적으로 문항의 수가 많고 점수 범주의 수가 적을 때 구성형 문항의 동등화 결과가 정확하였다.

다국어 초록 (Multilingual Abstract)

This simulation study evaluated the equating accuracy of constructed-response (CR) items by using IRT observed equating methods under the common-item nonequivalent groups design for the following conditions: 1) CR items were only used as common items with the unidimensional IRT models; 2) CR items and external SR items were used as common items with the unidimensional IRT models; 3) CR items and external SR items were used as common items with the multidimensional IRT models. The results indicated that accuracy of the CR item equating can be improved by adopting external SR items compared to equating only CR items under some restricted conditions, which are 1) multidimensional models, 2) large sample size, and 3) small group difference in SR items. In general, more items with fewer score categories lead to a greater equating accuracy for CR items.

목차 (Table of Contents)

• Ⅰ. Introduction Ⅱ. Method Ⅲ. Results Ⅳ. Discussion References

참고문헌 (Reference)

1 김성훈, "구성형문항 검사의 IRT 동등화를 위한 선다형문항 외부가교검사의 활용가능성 연구" 한국교육평가학회 22 (22): 195-218, 2009

2 황소림, "공통문항유형과 검사 차원성에 따른 혼합형 검사의 IRT 동등화" 한국교육평가학회 18 (18): 177-202, 2005

3 Fitzpatrick, A. R., "The effects of test length and sample size on the reliability and equating of tests composed of constructed-response items" 14 (14): 31-57, 2001

4 Kolen, M. J., "Test equating, scaling, and linking: Methods and practices" Springer 2014

5 Lim, E., "Subscore Equating and Profile Reporting" 33 (33): 95-112, 2020

6 Birnbaum, A., "Statistical Theories of Mental Test Scores" Addison-Wesley Publishing. 397-472, 1968

7 Kim, S., "Robustness to format effects of IRT linking methods for mixed format tests" 19 : 357-381, 2006

8 Kim, Y, K., "On Implications of a Hierarchical Rater/Signal Detection Model for Linking with Constructed Response Items" 2011

9 Yao, L., "Multidimensional Linking for Domain Scores and Overall Scores for Nonequivalent Groups" 35 (35): 48-66, 2011

10 Kim, S. Y., "Mixed-format tests: Psychometric properties with a primary focus on equating (volume 4)" CASMA, The University of Iowa 7-26, 2016

11 Hagge, S. L., "Mixed-format tests: Psychometric properties with a primary focus on equating (volume 2)" CASMA, The University of Iowa 45-86, 2012

12 Lee, W., "Mixed-format tests: Psychometric properties with a primary focus on equating (Volume 2)" Center for Advanced Studies in Measurement and Assessment, The University of Iowa 2012

13 Wang, W., "Mixed-format test score equating: effect of item-type multidimensionality, length and composition of common-item set, and group ability difference" University of Iowa. 2013

14 Kim, S., "Investigating the Effectiveness of Equating Designs for Constructed-Response Tests in Large-Scale Assessments" 47 (47): 186-201, 2010

15 Cai, L., "Generalized full-information item bifactor analysis" 16 (16): 221-, 2011

16 Hanson, B. A., "Extension of Lord-Wingersky algorithm to computing test score distributions for polytomous items"

17 Muraki, E., "Equating and linking of performance assessments" 24 (24): 325-337, 2000

18 McClellan, C. A., "Constructed-response scoring : Doing it right(ETS R&D Connections 13)" Educational Testing Service 2010

19 Lord, F. M., "Comparison of IRT true-score and observed score"equatings" 8 : 452-461, 1984

20 Zhang, J., "Calibration of response data using MIRT models with simple and mixed structures" 36 (36): 375-398, 2012

21 Lee, G., "Bi-factor MIRT observed-score equating for mixed-format tests" 29 (29): 224-241, 2016

22 Center for Applied Linguistics., "Annual technical report for ACCESS for ELLs® 2.0 Online English Language Proficiency Test, Series 501, 2018–2019 administration"

23 DeMauro, G. E., "An investigation of the appropriateness of the TOEFL test as a matching variable to equate TWE topics" Educational Testing Service 1992

24 Muraki, E., "A generalized partial credit model : Application of an EM algorithm" 16 : 159-177, 1992

25 Wang, T., "A comparison of the frequency estimation and chained equipercentile methods under the common-item nonequivalent groups design" 32 : 632-651, 2008

26 Huynh, H., "A comparison of equal percentile and partial credit equatings for performance-based assessments composed of free-response items" 31 : 125-141, 1994