4모수 문항반응모형에서 모수 추정의 정확성: Mplus와 R-mirt의 수행 비교 = The accuracy of parameter estimation in a 4-parameter IRT model: Comparison of Mplus and R-mirt

국문 초록 (Abstract)

본 연구에서는 4모수 로지스틱 문항반응모형의 문항 모수와 피험자 모수를 추정하는데 있어 두 추정 프로그램인 Mplus와 R패키지 mirt(이하 R-mirt)의 수행 정도를 비교하고자 하였다. 이를 위해 모의실험 연구를 실행하여 표본크기의 변화에 따른 문항과 피험자 모수 추정의 정확성을 bias와 root-mean-square error(RMSE)를 이용하여 탐색하였다. 표본크기의 조건은 5,000, 10,000, 15,000, 20,000, 25,000명으로 설정하였고, 문항 수는 40문항으로 고정하였다. 문항 모수 추정치에 대한 두 프로그램의 bias의 경우 뚜렷한 과대추정이나 과소추정의 경향이 보이지 않았고, RMSE는 표본크기가 커질수록 감소하는 패턴이 뚜렷하였으나 15,000명부터는 감소의 폭이 줄어들었다. 간소한 차이지만 모든 조건에서 Mplus가 R-mirt보다 더 작은 RMSE를 보였다. 피험자 모수 추정에서도 두 프로그램 간 수행 결과는 유사하였고, 양 극단의 능력 수준에서 bias와 RMSE 값이 가장 컸으며 0을 포함하는 중간 구간에서 가장 작았다. 본 연구는 4모수 모형의 추정을 위해 Mplus 또는 R-mirt의 수행 정도를 비교할 뿐 아니라, 연구자들이 두 프로그램을 실행하는데 필요한 정보와 가이드라인을 제공하고자 하였다.

다국어 초록 (Multilingual Abstract)

The purpose of the study is to compare the performances of the two programs, Mplus and an R package, mirt, in estimating the item and person parameters of a four-parameter logistic model. A simulation study was conducted, and the accuracy of the item and person parameters was explored using bias and RMSE under different sample size conditions. The conditions for sample size were set to 5,000, 10,000, 15,000, 20,000, and 25,000, and the number of items was fixed at 40. The bias of Mplus and R-mirt for the item parameter estimates did not show severe evidence of over- and under-estimation results. The RMSE tended to decrease as the sample size increased, but the decreasing rate was reduced from the condition of 15,000 sample size. Mplus showed lower RMSE values than R-mirt under all conditions, but the differences were trivial. In the estimation of person parameters, the results between the two programs were very similar. The values of bias and RMSE were large at the both extremes of ability levels. The smallest values of bias and RMSE were observed in the middle range containing 0. This study not only compared the performance of Mplus and R-mirt in estimating the four-parameter logistic model, but also intended to provide useful information and guidelines for researchers to implement Mplus or R-mirt in estimating the model.

목차 (Table of Contents)

• Ⅰ. 서 론 Ⅱ. 이론적 배경 Ⅲ. 연구 방법 Ⅳ. 연구 결과 Ⅴ. 결 론

참고문헌 (Reference)

1 김수영, "구조방정식 모형의 기본과 확장 : Mplus 예제와 함께" 학지사 2016

2 Partchev, I, "qirtoys: A Collection of Functions Related to Item Response Theory (IRT). R package version 0.2.0"

3 Chalmers, R. P., "mirt : A multidimensional item response theory package for the R environment" 48 (48): 1-29, 2012

4 Raîche, G, "irtProb: Utilities and Probability Distributions Related to Multidimensional Person Item Response Models. R package version 1.2"

5 Lunn, D. J., "WinBUGS-a Bayesian modeling framework : concepts, structure, and extensibility" 10 : 325-337, 2000

6 Merritt, J., "Why the folks at ETS flunked the coutse-a tech-savvy service will soon be giving B-school applicants their GMATs"

7 Primi, R., "Using Four-Parameter Item Response Theory to model Human Figure Drawings" 17 (17): 473-483, 2018

8 Liao, W. -W., "The four-parameter logistic item response theory model as a robust method of estimating ability despite aberrant responses" 40 : 1679-1694, 2012

9 Tang, K. L, "The effect of small calibration sample sizes on TOEFL IRT-based equating" Educational Testing Service 1993

10 The BUGS Project, "The BUGS Project Welcome Page"

11 Thissen, D., "Some standard errors in item response theory" 47 (47): 397-412, 1982

12 Seong, T. -J., "Sensitivity of marginal maximum likelihood estimation of item and ability parameters to the characteristics of the prior ability distributions" 14 : 299-311, 1990

13 Culpepper, S. A., "Revisiting The 4-parameter Item Response Model : Bayesian Estimation and Application" 81 (81): 1142-1163, 2016

14 Battauz, M., "Regularized Estimation of the Four-Parameter Logistic Model" 2 (2): 269-278, 2020

15 Hulin, C. L., "Recovery of two and three-parameter logistic item characteristic curves : A Monte Carlo study" 6 (6): 249-260, 1982

16 Goldman, S. H., "Recovery of one-and two-parameter logistic item parameters : An empirical study" 46 (46): 11-21, 1986

17 Stone, C. A., "Recovery of marginal maximum likelihood estimates in the two-parameter logistic response model : An evaluation of Multilog" 16 (16): 1-16, 1992

18 Wollack, J. A., "Recovery of item parameters in the nominal response model : A comparison of marginal maximum likelihood estimation and Markov chain Monte Carlo estimation" 26 (26): 339-352, 2002

19 Ihaka, R., "R : A language for data analysis and graphics" 5 (5): 299-314, 1996

20 Gourieroux, C., "Pseudo maximum likelihood methods : Theory" 52 : 681-700, 1984

21 Arminger, G., "Pseudo maximum likelihood estimation and a test for misspecification in mean and covariance structure models" 54 (54): 409-425, 1989

22 Huber, P. J., "Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability(Volume 1)" University of California Press. 221-233, 1967

23 Do, H., "Parameter Recovery for the Four- Parameter Unidimensional Binary IRT Model: A Comparison of Marginal Maximum Likelihood and Markov Chain Monte Carlo Approaches" Ohio University 2021

24 R. Philip Chalmers, "Numerical approximation of the observed information matrix with Oakes' identity" Wiley 71 (71): 415-436, 2018

25 McDonald, R. P, "Nonlinear Factor Analysis (Psychometric Monograph No. 15)" Psychometric Corporation 1967

26 Suh, Y., "Nested Logit Models for Multiple-Choice Item Response Data" 75 : 454-473, 2010

27 Muthén, L. K., "Mplus User’s Guide" Muthén & Muthén 2017

28 White, H., "Maximum likelihood estimation of misspecified models" 50 (50): 1-25, 1982

29 Meng, X., "Marginalized Maximum a Posteriori Estimation for the 4-Parameter Logistic Model under a Mixture Modeling Framework" 73 (73): 51-82, 2019

30 Bock, R. D., "Marginal maximum likelihood estimation of item parameters : Application of an EM algorithm" 46 : 443-459, 1981

31 Deonovic, B., "Learning meets Assessment : On the relation between Item Response Theory and Bayesian Knowledge Tracing" 45 : 457-474, 2018

32 Rulison, K. L., "I’ve fallen and I can’t get up: Can high ability students recover from early mistakes in computerized adaptive testing?" 33 : 83-101, 2009

33 Baker, F. B., "Item response theory. Parameter estimation techniques" Marcel Dekker 2004

34 Rupp, A. A., "Item response modeling with BILOG-MG and MULTILOG for Windows" 3 : 365-384, 2003

35 Chen, Y, "Item Response theory- A Statistical Framework for Educational and Psychological Measurement"

36 Asparouhov, T, "IRT in Mplus. Version 4"

37 Reise, S. P., "How many IRT parameters does it take to model psychopathology items" 8 : 164-184, 2003

38 Rusch, T., "Handbook of Modern Item Response Theory, Volume Three: Applications" Chapman and Hall/CRC Press. 407-420, 2018

39 Muthén, L. K., "Handbook of Item Response Theory, Volume Three: Applications" Chapman and Hall/CRC Press 507-517, 2018

40 Van der Linden, W. J., "Handbook of Item Response Theory, Volume Three : Applications" Chapman and Hall/CRC Press 2018

41 Lim, R. G., "Evaluation of two methods for estimating item response theory parameters when assessing differential item functioning" 75 (75): 164-174, 1990

42 Swaminathan, H., "Estimation of parameters in the three-parameter latent trait model" University of Massachusetts, School of Education, Laboratory of Psychometric and Evaluation Research 1979

43 Loken, E., "Estimation of a four-parameter item response theory model" 63 : 509-525, 2010

44 Hambleton, R. K., "Educational measurement" Macmillan Publishing. 147-200, 1989

45 Linacre, J. M., "Discrimination, guessing and Careless: Estimation IRT parameters with Rasch" 18 : 959-960, 2004

46 Mahmood Ul Hassan, "Discrimination with unidimensional and multidimensional item response theory models for educational data" Informa UK Limited 1-21, 2019

47 Barnard-Brak, L., "Differences in mathematics achievement according to opportunity to learn : A 4pl item response theory examination" 56 : 1-7, 2018

48 Magis, D., "Computerized adaptive and multistage testing with R: Using packages catR and mstR" Springer International Publishing 2017

49 Ma, W., "Choosing between CDM and Unidimensional IRT : The Proportional Reasoning Test Case, Measurement" 18 (18): 87-96, 2020

50 Roberts, J. S., "Characteristics of MML/EAP parameter estimates in the generalized graded unfolding model" 26 : 192-207, 2002

51 Waller, N., "Bayesian Modal Estimation of the Four-Parameter Item Response Model in Real, Realistic, and Idealized Data Sets" 52 (52): 1-21, 2017

52 Ogasawara, H., "Asymptotic expansions for the ability estimator in item response theory" 27 : 661-683, 2012

53 Meyer, J. P., "Applied measurement with jMetrik" Routledge 2014

54 Osgood, D., "Analyzing multiple-item measures of crime and deviance I : Item response theory scaling" 18 : 267-296, 2002

55 Barton, M. A., "An upper asymptote for the three-parameter logistic item-response model" Educational Testing Service 1981

56 Kalkan, Ö. K., "An evaluation of 4PL IRT and DINA models for estimating pseudo-guessing and slipping parameters" 11 (11): 131-146, 2020

57 Harwell, M. R., "An empirical study of the effects of small datasets and varying prior variances on item parameter estimation in BILOG" 15 (15): 279-291, 1991

58 Lord, F. M., "An analysis of the verbal scholastic aptitude test using Birnbaum’s three parameter logistic model" 28 (28): 989-1020, 1968

59 Yen, Y. -C., "An Empirical Evaluation of the Slip Correction in the Four Parameter Logistic Models With Computerized Adaptive Testing" 36 (36): 75-87, 2012

60 Linacre, J. M., "A user's guide to WINSTEPS/ MINISTEPS Rasch-model computer programs : Program manual 3. 71. 0" Winsteps.com. 2010

61 Meyer, J. P., "A study or Rasch, partial credit, and rating scale model parameter recovery in WINSTEPS and jMetrik" 13 (13): 248-258, 2012

62 Yen, W. M., "A comparison of the efficiency and accuracy of Bilog and Logist" 52 (52): 275-291, 1987

63 Patsula, L. N., "A comparison of item parameter estimates and ICCs produced with TESTGRAF and BILOG under different test lengths and sample sizes" 1995

64 David Magis, "A Note on the Item Information Function of the Four-Parameter Logistic Model" SAGE Publications 37 (37): 304-315, 2013

65 안선영, "4모수 문항반응모형을 적용한 TIMSS 2015 수학 검사의 문항모수 추정" 한국교육평가학회 34 (34): 231-256, 2021