사회과학 대용량 자료 분석을 위한 벌점회귀모형과 결측처리기법의 성능 비교: 몬테카를로 모의실험

노민정,유진은 한국교육평가학회 2019 교육평가연구 Vol.32 No.4

The purpose of this Monte-Carlo simulation study was to investigate missing data techniques and penalized regression methods for social science large-scale data. Data of 340 variables and 2,000 observations were generated to emulate a social science panel data, KCYPS(Korean Children and Youth Panel Survey). The simulation conditions included missingness mechanisms (MAR, MNAR), missing data techniques (listwise deletion, k-NN, EM), and penalied regression methods (LASSO, adaptive LASSO, and MCP). As a result, the simulation had 18 condition combinations, and each condition had 100 replications. For evaluation criteria, agreement rates were used for the performance of missing data techniques and IC1 and IC2 were used for variable selection. Prediction accuracy, AUC, and Kappa were utilized for model evaluation criteria. With regard to missing data imputation, k-nn outperformed EM. Listwise deletion deteriorated the performance of penalized regression. LASSO and adaptive LASSO tended to outperform MCP in terms of variable selection and prediction. Further research topics were discussed accordingly. 본 몬테카를로 모의실험의 목적은 사회과학 분야의 대용량 자료 분석에 적합한 결측처리기법과 벌점회귀모형을 파악하는 것이었다. 먼저 KCYPS(Korean Chidren and Youth Panel Survey) 자료의 특징을 모방하여 340개 변수와 2,000명 자료를 생성하고 MAR 또는 MNAR 결측 메커니즘으로 결측을 발생시켰다. 다음으로 완전제거법, k-NN 대체법, 또는 EM 알고리즘 대체법으로 결측을 대체하고, 대체된 자료에 LASSO, adaptive LASSO, 또는 MCP 벌점회귀모형을 적용하였다. 결측 메커니즘(MAR, MNAR), 결측처리기법(완전제거법, k-NN, EM), 그리고 벌점회귀모형(LASSO, adaptive LASSO, MCP)으로 구성된 총 18개 조건에 대하여 100번 반복한 몬테카를로 모의실험을 실시하여 결측 대체, 변수 선택, 그리고 예측 성능을 비교하였다. 모형 평가 기준으로 결측 대체에 있어 일치율, 변수 선택의 경우 IC1, IC2, 그리고 예측 성능에 있어 정확도, AUC, Kappa 계수를 활용하였다. 연구 결과, 결측 대체에 있어서는 k-NN이 EM보다 우수하였으며, 완전제거법은 벌점회귀모형의 성능을 크게 저하시키는 것으로 확인되었다. MCP와 비교 시 LASSO와 adaptive LASSO의 변수 선택 및 예측 성능이 좋은 편이었다. 연구 결과를 바탕으로 후속 연구에 대하여 제언하였다.

Variable Selection Via Penalized Regression

Yoon, Young-Joo,Song, Moon-Sup The Korean Statistical Society 2005 Communications for statistical applications and me Vol.12 No.3

In this paper, we review the variable-selection properties of LASSO and SCAD in penalized regression. To improve the weakness of SCAD for high noise level, we propose a new penalty function called MSCAD which relaxes the unbiasedness condition of SCAD. In order to compare MSCAD with LASSO and SCAD, comparative studies are performed on simulated datasets and also on a real dataset. The performances of penalized regression methods are compared in terms of relative model error and the estimates of coefficients. The results of experiments show that the performance of MSCAD is between those of LASSO and SCAD as expected.

Principal Component Regression by Principal Component Selection

Lee, Hosung,Park, Yun Mi,Lee, Seokho The Korean Statistical Society 2015 Communications for statistical applications and me Vol.22 No.2

We propose a selection procedure of principal components in principal component regression. Our method selects principal components using variable selection procedures instead of a small subset of major principal components in principal component regression. Our procedure consists of two steps to improve estimation and prediction. First, we reduce the number of principal components using the conventional principal component regression to yield the set of candidate principal components and then select principal components among the candidate set using sparse regression techniques. The performance of our proposals is demonstrated numerically and compared with the typical dimension reduction approaches (including principal component regression and partial least square regression) using synthetic and real datasets.

Semiparametric kernel Logistic regression with Longitudinal data

Joo Yong Shim,Kyung Ha Seok 한국데이터정보과학회 2012 한국데이터정보과학회지 Vol.23 No.2

Logistic regression is a well known binary classi cation method in the field of sta-tistical learning. Mixed-effect regression models are widely used for the analysis of correlated data such as those found in longitudinal studies. We consider kernel extensions with semiparametric fixed effects and parametric random effects for the logistic regression. The estimation is performed through the penalized likelihood method based on kernel trick, and our focus is on the efficient computation and the effective hyperpar-ameter selection. For the selection of optimal hyperparameters, cross-validation tech-niques are employed. Numerical results are then presented to indicate the performance of the proposed procedure.

Semiparametric kernel logistic regression with longitudinal data

Shim, Joo-Yong,Seok, Kyung-Ha The Korean Data and Information Science Society 2012 한국데이터정보과학회지 Vol.23 No.2

Logistic regression is a well known binary classification method in the field of statistical learning. Mixed-effect regression models are widely used for the analysis of correlated data such as those found in longitudinal studies. We consider kernel extensions with semiparametric fixed effects and parametric random effects for the logistic regression. The estimation is performed through the penalized likelihood method based on kernel trick, and our focus is on the efficient computation and the effective hyperparameter selection. For the selection of optimal hyperparameters, cross-validation techniques are employed. Numerical results are then presented to indicate the performance of the proposed procedure.

Semiparametric kernel logistic regression with longitudinal data

심주용,석경하 한국데이터정보과학회 2012 한국데이터정보과학회지 Vol.23 No.2

Logistic regression is a well known binary classication method in the field of statistical learning. Mixed-effect regression models are widely used for the analysis of correlated data such as those found in longitudinal studies. We consider kernel extensions with semiparametric fixed effects and parametric random effects for the logistic regression. The estimation is performed through the penalized likelihood method based on kernel trick, and our focus is on the efficient computation and the effective hyperparameter selection. For the selection of optimal hyperparameters, cross-validation techniques are employed. Numerical results are then presented to indicate the performance of the proposed procedure.

Kernel Poisson regression for mixed input variables

Joo Yong Shim 한국데이터정보과학회 2012 한국데이터정보과학회지 Vol.23 No.6

An estimating procedure is introduced for kernel Poisson regression when the input variables consist of numerical and categorical variables, which is based on the penalized negative log-likelihood and the component-wise product of two different types of kernel functions. The proposed procedure provides the estimates of the mean function of the response variables, where the canonical parameter is linearly and /or nonlinearly related to the input variables. Experimental results are then presented which indicate the performance of the proposed kernel Poisson regression.

Kernel Poisson regression for mixed input variables

Shim, Jooyong The Korean Data and Information Science Society 2012 한국데이터정보과학회지 Vol.23 No.6

An estimating procedure is introduced for kernel Poisson regression when the input variables consist of numerical and categorical variables, which is based on the penalized negative log-likelihood and the component-wise product of two different types of kernel functions. The proposed procedure provides the estimates of the mean function of the response variables, where the canonical parameter is linearly and/or nonlinearly related to the input variables. Experimental results are then presented which indicate the performance of the proposed kernel Poisson regression.

Marginalized lasso in sparse regression

이석호,김선화 한국통계학회 2019 Journal of the Korean Statistical Society Vol.48 No.3

We propose marginalized lasso, a new nonconvex penalization for variable selection in regression problem. The marginalized lasso penalty is motivated from integrating out the penalty parameter in the original lasso penalty with a gamma prior distribution. This study provides a thresholding rule and a lasso-based iterative algorithm for parameter estimation in the marginalized lasso. We also provide a coordinate descent algorithm to efficiently optimize the marginalized lasso penalized regression. Numerical comparison studies are provided to demonstrate its competitiveness over the existing sparsity-inducing penalizations and suggest some guideline for tuning parameter selection.

Kernel Poisson regression for mixed input variables

심주용 한국데이터정보과학회 2012 한국데이터정보과학회지 Vol.23 No.6

An estimating procedure is introduced for kernel Poisson regression when the input variables consist of numerical and categorical variables, which is based on the penalized negative log-likelihood and the component-wise product of two different types of kernel functions. The proposed procedure provides the estimates of the mean function of the response variables, where the canonical parameter is linearly and/or nonlinearly related to the input variables. Experimental results are then presented which indicate the performance of the proposed kernel Poisson regression.