베이즈 정보 기준을 활용한 분할-정복 벌점화 분위수 회귀 = Model selection via Bayesian information criterion for divide-and-conquer penalized quantile regression

다국어 초록 (Multilingual Abstract)

Quantile regression is widely used in many fields based on the advantage of providing an efficient tool for examining complex information latent in variables. However, modern large-scale and high-dimensional data makes it very difficult to estimate the quantile regression model due to limitations in terms of computation time and storage space. Divide-and-conquer is a technique that divide the entire data into several sub-datasets that are easy to calculate and then reconstruct the estimates of the entire data using only the summary statistics in each sub-datasets. In this paper, we studied on a variable selection method using Bayes information criteria by applying the divide-and-conquer technique to the penalized quantile regression. When the number of sub-datasets is properly selected, the proposed method is efficient in terms of computational speed, providing consistent results in terms of variable selection as long as classical quantile regression estimates calculated with the entire data. The advantages of the proposed method were confirmed through simulation data and real data analysis.

국문 초록 (Abstract)

분위수 회귀 모형은 변수에 숨겨진 복잡한 정보를 살펴보기 위한 효율적인 도구를 제공하는 장점을 바탕으로 많은 분야에서 널리 사용되고 있다. 그러나 현대의 대용량-고차원 데이터는 계산 시간 및 저장공간의 제한으로 인해 분위수 회귀 모형의 추정을 매우 어렵게 만든다. 분할-정복은 전체 데이터를 계산이 용이한 여러개의 부분집합으로 나눈 다음 각 분할에서의 요약 통계량만을 이용하여 전체 데이터의 추정량을 재구성하는 기법이다. 본 연구에서는 분할-정복 기법을 벌점화 분위수 회귀에 적용하고 베이즈 정보기준을 활용하여 변수를 선택하는 방법에 관하여 연구하였다. 제안 방법은 분할 수를 적절하게 선택하였을 때, 전체 데이터로 계산한 일반적인 분위수 회귀 추정량만큼 변수 선택의 측면에서 일관된 결과를 제공하면서 계산 속도의 측면에서 효율적이다. 이러한 제안된 방법의 장점은 시뮬레이션 데이터 및 실제 데이터 분석을 통해 확인하였다.

참고문헌 (Reference)

1 정병호 ; 임동훈, "빅데이터 통합모형 비교분석" 한국데이터정보과학회 28 (28): 755-768, 2017

2 방성완 ; 신승준, "비교차 제약식을 이용한 다중 선형 분위수 회귀모형에 관한 비교연구" 한국통계학회 29 (29): 773-786, 2016

3 강종경 ; 전명식, "대용량 자료의 분석을 위한 분할정복 랜덤스케치 커널 능형회귀" 한국데이터정보과학회 31 (31): 15-23, 2020

4 Srivastava S, "WASP: Scalable Bayes via barycenters of subset posteriors" 912-920, 2015

5 Fan J, "Variable selection via nonconcave penalized likelihood and its oracle properties" 96 : 1348-1360, 2001

6 Wu Y, "Variable selection in quantile regression" 801-817, 2009

7 Wang H, "Unified Lasso estimation by least squares approximation" 102 : 1418-1429, 2007

8 Wang H, "Tuning parameter Selectors for the Smoothly Clipped Absolute Deviation Method" 94 : 553-568, 2007

9 Bertin-Mahieux T, "The million song dataset" 2011

10 Powell D, "The exporter productivity premium along the productivity distribution : evidence from quantile regression with nonadditive firm fixed effects" 150 : 763-785, 2014

11 Okada K, "The effect of foreign aid on corruption : a quantile regression approach" 115 : 240-243, 2012

12 Portnoy S, "The Gaussian hare and the Laplacian tortoise : computability of squared-error versus absolute-error estimators" 12 : 279-300, 1997

13 Li R, "Statistical inference in massive data sets" 29 : 399-409, 2013

14 Cole T, "Smoothing reference centile curves : The LMS method and penalized likelihood" 11 : 1305-1319, 1992

15 Jiang R, "Single-index composite quantile regression with heteroscedasticity and general error distributions" 57 : 185-203, 2016

16 Wang H, "Shrinkage tuning parameter selection with a diverging number of parameters" 71 : 671-683, 2009

17 Heagerty P, "Semiparametric estimation of regression quantiles with application to standardizing weight for height and age in U.S. children" 48 : 533-551, 1999

18 Dasgupta A, "Sampling algorithms and corsets for lp regression" 38 : 2060-2078, 2009

19 Tibshirani R, "Regression shrinkage and selection via the LASSO" 58 : 267-288, 1996

20 Fan TH, "Regression analysis for massive datasets" 61 : 554-562, 2007

21 Chen L, "Quantile regression in big data : A divide and conquer based strategy" 144 : 106892-, 2020

22 Yang J, "Quantile regression for large-scale applications" 36 : 78-110, 2014

23 Wang L, "Quantile regression for analyzing Heterogeneity in Ultra-High Dimension" 107 : 214-222, 2012

24 Lee E, "Model selection via Bayesian information criterion for quantile regression models" 109 : 216-229, 2014

25 Alhamzawi R, "Model selection in quantile regression models" 42 : 445-458, 2015

26 Belloni A, "L1-penalized quantile regression in high-dimensional sparse models" 39 : 82-130, 2011

27 Li Y, "L1-norm quantile regression" 17 : 163-185, 2008

28 Cook L, "Gendered parenthood penalties and premiums across the earnings distribution in Australia, the United Kingdom, and the United States" 30 : 360-372, 2014

29 Ning Z, "Estimation and test procedures for composite quantile regression with covariates missing at random" 95 : 15-25, 2014

30 Xue J, "Double-parallel Monte Carlo for Bayesian analysis of big data" 29 : 23-32, 2019

31 Zhang Y, "Divide and conquer kernel ridge regression : A distributed algorithm with minimax optimal rates" 16 : 3299-3340, 2015

32 Scott SL, "Comparing consensus Monte Carlo strategies for distributed Bayesian computation" 31 : 668-685, 2017

33 Xu Q, "Block average quantile regression for massive dataset" 61 : 141-165, 2020

34 Scott SL, "Bayes and big data : The consensus Monte Carlo algorithm" 11 : 78-88, 2016

35 Peng B, "An iterative coordinate descent algorithm for high-dimensional nonconvex penalized quantile regression" 24 : 676-694, 2015

36 김경희 ; 신승준, "Adaptive ridge procedure for L0-penalized weighted support vector machine" 한국데이터정보과학회 28 (28): 1271-1278, 2017

37 Frank I, "A statistical view of some chemometrics regression tools" 35 : 109-148, 1993

38 Chen X, "A split-and-conquer approach for analysis of extraordinarily large data" 165-1684, 2014

39 Koenker R, "A Frisch-Newton algorithm for sparse quantile regression" 21 : 225-236, 2005