유전 알고리즘을 이용한 국소가중회귀의 다중모델 결합을 위한 점진적 앙상블 학습

김상훈 ( Kim Sang Hun ),정병희 ( Chung Byung Hee ),이건호 ( Lee Gun Ho ) 한국정보처리학회 2018 정보처리학회논문지. 소프트웨어 및 데이터 공학 Vol.7 No.9

전통적으로 나태한 학습에 해당하는 국소가중회귀(LWR: Locally Weighted Regression)모델은 입력변수인 질의지점에 따라 예측의 해를 얻기 위해 일정구간 범위내의 학습 데이터를 대상으로 질의지점의 거리에 따라 가중값을 달리 부여하여 학습 한 결과로 얻은 짧은 구간내의 회귀식이다. 본 연구는 메모리 기반학습의 형태에 해당하는 LWR을 위한 점진적 앙상블 학습과정을 제안한다. LWR를 위한 본 연구의 점진적 앙상블 학습법은 유전알고리즘을 이용하여 시간에 따라 LWR모델들을 순차적으로 생성하고 통합하는 것이다. 기존의 LWR 한계는 인디케이터 함수와 학습 데이터의 선택에 따라 다중의 LWR모델이 생성될 수 있으며 이 모델에 따라 예측 해의 질도 달라질 수 있다. 하지만 다중의 LWR 모델의 선택이나 결합의 문제 해결을 위한 연구가 수행되지 않았다. 본 연구에서는 인디케이터 함수와 학습 데이터에 따라 초기 LWR 모델을 생성한 후 진화 학습 과정을 반복하여 적절한 인디케이터 함수를 선택하며 또한 다른 학습 데이터에 적용한 LWR 모델의 평가와 개선을 통하여 학습 데이터로 인한 편향을 극복하고자 한다. 모든 구간에 대해 데이터가 발생 되면 점진적으로 LWR모델을 생성하여 보관하는 열심학습(Eager learning)방식을 취하고 있다. 특정 시점에 예측의 해를 얻기 위해 일정구간 내에 신규로 발생된 데이터들을 기반으로 LWR모델을 생성한 후 유전자 알고리즘을 이용하여 구간 내의 기존 LWR모델들과 결합하는 방식이다. 제안하는 학습방법은 기존 단순평균법을 이용한 다중 LWR모델들의 선택방법 보다 적합도 평가에서 우수한 결과를 보여주고 있다. 특정지역의 시간 별 교통량, 고속도로 휴게소의 시간별 매출액 등의 실제 데이터를 적용하여 본 연구의 LWR에 의한 결과들의 연결된 패턴과 다중회귀분석을 이용한 예측결과를 비교하고 있다. The LWR (Locally Weighted Regression) model, which is traditionally a lazy learning model, is designed to obtain the solution of the prediction according to the input variable, the query point, and it is a kind of the regression equation in the short interval obtained as a result of the learning that gives a higher weight value closer to the query point. We study on an incremental ensemble learning approach for LWR, a form of lazy learning and memory-based learning. The proposed incremental ensemble learning method of LWR is to sequentially generate and integrate LWR models over time using a genetic algorithm to obtain a solution of a specific query point. The weaknesses of existing LWR models are that multiple LWR models can be generated based on the indicator function and data sample selection, and the quality of the predictions can also vary depending on this model. However, no research has been conducted to solve the problem of selection or combination of multiple LWR models. In this study, after generating the initial LWR model according to the indicator function and the sample data set, we iterate evolution learning process to obtain the proper indicator function and assess the LWR models applied to the other sample data sets to overcome the data set bias. We adopt Eager learning method to generate and store LWR model gradually when data is generated for all sections. In order to obtain a prediction solution at a specific point in time, an LWR model is generated based on newly generated data within a predetermined interval and then combined with existing LWR models in a section using a genetic algorithm. The proposed method shows better results than the method of selecting multiple LWR models using the simple average method. The results of this study are compared with the predicted results using multiple regression analysis by applying the real data such as the amount of traffic per hour in a specific area and hourly sales of a resting place of the highway, etc.

A New Incremental Learning Algorithm with Probabilistic Weights Using Extended Data Expression

Yang, Kwangmo,Kolesnikova, Anastasiya,Lee, Won Don The Korea Institute of Information and Commucation 2013 Journal of information and communication convergen Vol.11 No.4

New incremental learning algorithm using extended data expression, based on probabilistic compounding, is presented in this paper. Incremental learning algorithm generates an ensemble of weak classifiers and compounds these classifiers to a strong classifier, using a weighted majority voting, to improve classification performance. We introduce new probabilistic weighted majority voting founded on extended data expression. In this case class distribution of the output is used to compound classifiers. UChoo, a decision tree classifier for extended data expression, is used as a base classifier, as it allows obtaining extended output expression that defines class distribution of the output. Extended data expression and UChoo classifier are powerful techniques in classification and rule refinement problem. In this paper extended data expression is applied to obtain probabilistic results with probabilistic majority voting. To show performance advantages, new algorithm is compared with Learn++, an incremental ensemble-based algorithm.

Ensemble techniques and hybrid intelligence algorithms for shear strength prediction of squat reinforced concrete walls

Mohammad Sadegh Barkhordari,Leonardo M. Massone Techno-Press 2023 Advances in computational design Vol.8 No.1

Squat reinforced concrete (SRC) shear walls are a critical part of the structure for both office/residential buildings and nuclear structures due to their significant role in withstanding seismic loads. Despite this, empirical formulae in current design standards and published studies demonstrate a considerable disparity in predicting SRC wall shear strength. The goal of this research is to develop and evaluate hybrid and ensemble artificial neural network (ANN) models. State-of-the-art population-based algorithms are used in this research for hybrid intelligence algorithms. Six models are developed, including Honey Badger Algorithm (HBA) with ANN (HBA-ANN), Hunger Games Search with ANN (HGS-ANN), fitness-distance balance coyote optimization algorithm (FDB-COA) with ANN (FDB-COA-ANN), Averaging Ensemble (AE) neural network, Snapshot Ensemble (SE) neural network, and Stacked Generalization (SG) ensemble neural network. A total of 434 test results of SRC walls is utilized to train and assess the models. The results reveal that the SG model not only minimizes prediction variance but also produces predictions (with R²= 0.99) that are superior to other models.

물체 추적을 위한 딥 러닝 기반의 앙상블 모델 연구

김민지(Minji Kim),정일채(Ilchae Jung),한보형(Bohyung Han) Korean Institute of Information Scientists and Eng 2021 정보과학회논문지 Vol.48 No.2

In the area of computer vision, visual object tracking aims to estimate the status of a target object from an input video stream, which can be broadly applicable to industries such as surveillance and the military. Recently, deep learning-based tracking algorithms have gone through significant improvements by using tracking-by-detection or template-based approach. However, these approaches are still suffering from inherent limitations caused by each strategy. In this paper, we propose a novel method to model ensemble trackers by fusing the two strategies, tracking-by-detection and template-based approach. We report significantly enhanced performance on widely adopted visual object tracking benchmarks, OTB100, UAV123, and LaSOT.

Harvest Forecasting Improvement Using Federated Learning and Ensemble Model

Ohnmar Khin,고진광,이성근 (사)한국스마트미디어학회 2023 스마트미디어저널 Vol.12 No.10

Harvest forecasting is the great demand of multiple aspects like temperature, rain, environment, and their relations. The existing study investigates the climate conditions and aids the cultivators to know the harvest yields before planting in farms. The proposed study uses federated learning. In addition, the additional widespread techniques such as bagging classifier, extra tees classifier, linear discriminant analysis classifier, quadratic discriminant analysis classifier, stochastic gradient boosting classifier, blending models, random forest regressor, and AdaBoost are utilized together. These presented nine algorithms achieved exemplary satisfactory accuracies. The powerful contributions of proposed algorithms can create exact harvest forecasting. Ultimately, we intend to compare our study with the earlier research's results.

Hybrid Feature Selection Method Based on Genetic Algorithm for the Diagnosis of Coronary Heart Disease

Wiharto Wiharto,Esti Suryani,Sight Setywan,Bintang PE Putra 한국정보통신학회 2022 Journal of information and communication convergen Vol.20 No.1

Coronary heart disease (CHD) is a comorbidity of COVID-19; therefore, routine early diagnosis is crucial. A large number of examination attributes in the context of diagnosing CHD is a distinct obstacle during the pandemic when the number of health service users is significant. The development of a precise machine learning model for diagnosis with a minimum number of examination attributes can allow examinations and healthcare actions to be undertaken quickly. This study proposes a CHD diagnosis model based on feature selection, data balancing, and ensemble-based classification methods. In the feature selection stage, a hybrid SVM-GA combined with fast correlation-based filter (FCBF) is used. The proposed system achieved an accuracy of 94.60% and area under the curve (AUC) of 97.5% when tested on the z-Alizadeh Sani dataset and used only 8 of 54 inspection attributes. In terms of performance, the proposed model can be placed in the very good category.

앙상블 학습 알고리즘을 이용한 컨벌루션 신경망의 분류 성능 분석에 관한 연구

박성욱,김종찬,김도연 한국멀티미디어학회 2019 멀티미디어학회논문지 Vol.22 No.6

In this paper, we compare and analyze the classification performance of deep learning algorithm Convolutional Neural Network(CNN) ac cording to ensemble generation and combining techniques. We used several CNN models(VGG16, VGG19, DenseNet121, DenseNet169, DenseNet201, ResNet18, ResNet34, ResNet50, ResNet101, ResNet152, GoogLeNet) to create 10 ensemble generation combinations and applied 6 combine techniques(average, weighted average, maximum, minimum, median, product) to the optimal combination. Experimental results, DenseNet169-VGG16-GoogLeNet combination in ensemble generation, and the product rule in ensemble combination showed the best performance. Based on this, it was concluded that ensemble in different models of high benchmarking scores is another way to get good results.

PreVision : 플랜트의 이상 징후를 조기에 감지하기 위한 솔루션

김현식(HyunSik Kim),유준우(Junwoo Yoo),박진성(JinSung Park) 대한기계학회 2022 대한기계학회 춘추학술대회 Vol.2022 No.11

In this paper, based on a machine learning algorithm, we propose a fault detection method that can predict fault and detect it early. This method has the advantage of having high accuracy and versatility at the same time, and it consists of Ensemble Learning based on a prediction algorithm to predict fault at an early stage and Auto-Learning Algorithm for automatically optimizing the learning model. By applying the aforementioned algorithm to a solution called PreVision, it was confirmed that anomalies were detected early in various plants. PreVision can utilize its powerful simulation capabilities to validate results on its own.

PreVision : 플랜트의 이상 징후를 조기에 감지하기 위한 솔루션

김현식(HyunSik Kim),유준우(Junwoo Yoo),박진성(JinSung Park) 대한기계학회 2022 대한기계학회 춘추학술대회 Vol.2022 No.11

In this paper, based on a machine learning algorithm, we propose a fault detection method that can predict fault and detect it early. This method has the advantage of having high accuracy and versatility at the same time, and it consists of Ensemble Learning based on a prediction algorithm to predict fault at an early stage and Auto-Learning Algorithm for automatically optimizing the learning model. By applying the aforementioned algorithm to a solution called PreVision, it was confirmed that anomalies were detected early in various plants. PreVision can utilize its powerful simulation capabilities to validate results on its own.

Ensemble variable selection using genetic algorithm

Seogyoung, Lee,Martin Seunghwan, Yang,Jongkyeong, Kang,Seung Jun, Shin The Korean Statistical Society 2022 Communications for statistical applications and me Vol.29 No.6

Variable selection is one of the most crucial tasks in supervised learning, such as regression and classification. The best subset selection is straightforward and optimal but not practically applicable unless the number of predictors is small. In this article, we propose directly solving the best subset selection via the genetic algorithm (GA), a popular stochastic optimization algorithm based on the principle of Darwinian evolution. To further improve the variable selection performance, we propose to run multiple GA to solve the best subset selection and then synthesize the results, which we call ensemble GA (EGA). The EGA significantly improves variable selection performance. In addition, the proposed method is essentially the best subset selection and hence applicable to a variety of models with different selection criteria. We compare the proposed EGA to existing variable selection methods under various models, including linear regression, Poisson regression, and Cox regression for survival data. Both simulation and real data analysis demonstrate the promising performance of the proposed method.