Accuracy Estimation of a Classifier Based on the Differences in Samples

Min Zhang,Shengbo Yu 보안공학연구지원센터 2016 International Journal of Signal Processing, Image Vol.9 No.11

The classification accuracy is an important standard to measure the quality of the classifier. Usually, the classification accuracy is assessed later, not during the classification process. Problems such as classification accuracy drops cannot be timely and effectively found. It is necessary that marking test samples when estimating classification accuracy. It is a problem that we care about that how much is the classification accuracy when a group of new samples obtained. The problem must be concerned when using and improving the classifier in the case of growing data. To solve this problem, this paper put forward different estimates from different perspectives which based on the difference between samples. One estimate is based on the difference in samples distribution, which is from the Bayesian criterion. Another estimate is based on the difference in each sample instance, which is from the K nearest neighbor classification. Classification accuracy is also estimated by using the artificial neural networks, which combine the characteristics of the above two methods. And results show the proposed methods have good effects.

Symmetrically Relative Accuracy as Rule Evaluation Measure

박희창 한국자료분석학회 2014 Journal of the Korean Data Analysis Society Vol.16 No.4

Rule evaluation is one of the most popular techniques and plays an important role in data mining. A lot of measures have been proposed in different fields that try to evaluate features of the rules obtained by different types of association and classification tasks. The most frequently used rule evaluation measures are rule accuracy, sensitivity, relative accuracy, and relative sensitivity. In this paper, we propose a symmetrically relative accuracy to compensate the shortcomings of rule accuracy, sensitivity, relative accuracy, and relative sensitivity. This measure is divided the sum of rule accuracy and sensitivity to the sum of relative accuracy and relative sensitivity. It is a symmetric measure whose value is not changed by the position of antecedent and consequent item. And then we investigated the conditions of interestingness measures by Piatetsky-Shapiro (1991), and compared some properties of rule accuracy, sensitivity, relative accuracy, relative sensitivity, and symmetrically relative accuracy. The results showed that the symmetrically relative accuracy was more meaningful rule evaluation measure because it was symmetric, and monotonically increased as co-occurrence frequency increased.

노화와 과제의 정확성 제약이 연속적인 조준 움직임 수행에 미치는 영향

이지항 ( Ji-hang Lee ),송용관 ( Yong-gwan Song ) 한국스포츠심리학회 2017 한국스포츠심리학회지 Vol.28 No.3

목적: 이 연구는 노화와 과제의 정확성 제약이 연속적인 조준 움직임 계획과 실행에 어떻게 영향을 미치는지를 규명하였다. 방법: 건강한 노인 11명을 노인집단으로 그리고 성인 11명은 성인집단으로 구분하여 3가지의 정확성 제약(난이도)을 지닌 Fitts 과제를 수행하게 하였다. Fitts 과제는 낮은 정확성 제약, 높은 정확성 제약 그리고 결합된 정확성 제약을 지닌 과제이며, 이 과제는 가상현실을 통해 컴퓨터로 구현되어 물체의 조작이나 외부적인 마찰 없이 움직임 제스처에 의해서 동작을 수행한다. 노화와 과제의 정확성 제약에 따른 연속적인 조준 움직임 계획과 실행에서의 차이를 확인하기 위해 과제 수행에 대한 움직임 시간과 체류 시간을 운동학적 자료로 수집하였다. 결과: 연구결과, 첫째, 정확성 제약이 높은 과제를 수행할 때, 노인들의 움직임 시간은 성인들에 비해 상대적으로 긴 것으로 나타났으며, 통계적으로 유의한 차이가 나타났다. 둘째, 체류시간에 있어서도 노인들은 정확성 제약이 높은 과제를 수행할 때, 목표 표적에 머무는 시간이 상대적으로 긴 것으로 드러났으며, 통계적으로 유의한 차이가 나타났다. 그러나 상대적으로 정확성 제약이 낮은 과제에서는 노인과 성인간의 차이가 확인되지 않았다. 결론: 이 연구를 통해 연속적인 조준 움직임 수행 시 노화는 감각정보의 지각 및 통합에서의 지연을 유발하여 연속적인 조준 움직임 수행을 방해할 수 있다는 것을 시사한다. Purpose: The present study examined the effects of accuracy constraint and aging on movement performance during continuous aiming tasks. Methods: Eleven older adults and 11 young adults were instructed to perform repetitive aiming movement to specified targets under three-accuracy constraints: 1) low accuracy (W=2cm) - minimal accuracy constraint, 2) high accuracy (W=0.64cm) - maximum accuracy constraint, and 3) mixed accuracy constraint - one target of high accuracy and another target of low accuracy. Kinematic data was collected by movement time and dwell time. Results: The results showed that movement time of older adults was increased across accuracy constraint compared to young adults. Dwell time of older adults also were longer than young adults with accuracy constraint. Conclusion: These results suggest that aging may induce deficits in ongoing planning and organizing processes during movement execution when the tasks require to assemble various accuracy requirements into more complex movement sequences.

국토관측위성용 정밀영상생성시스템의 위치정확도 분석

이유진,박형준,김혜성,김태정 대한원격탐사학회 2020 大韓遠隔探査學會誌 Vol.36 No.5

This paper reports on the analysis of the location accuracy of a precision image generation system manufactured for CAS 500. The planned launch date of the CAS 500 is 2021, and since it has not yet been launched, the analysis was performed using KOMPSAT-3A satellite images having similar specifications to the CAS 500. In this paper, we have checked the geolocation accuracy of initial sensor model, the model point geolocation accuracy of the precise sensor model, the geolocation accuracy of the precise sensor model using the check point, and the geolocation accuracy of the precise orthoimage using 30 images of the Korean Peninsula. In this study, the target geolocation accuracy is to have an RMSE within 2 pixels when an accurate ground control point is secured. As a result, it was confirmed that the geolocation accuracy of the precision sensor model using the checkpoint was about 1.85 pixels in South Korea and about 2.04 pixels in North Korea, and the geolocation accuracy of the precise orthoimage was about 1.15 m in South Korea and about 3.23 m in North Korea. Overall, it was confirmed that the accuracy of North Korea was low compared to that of South Korea, and this was confirmed to have affected the measured accuracy because the GCP (Ground Control Point) quality of the North Korea images was poor compared to that of South Korea. In addition, it was confirmed that the accuracy of the precision orthoimage was slightly lower than that of precision sensor medel, especially in North Korea. It was judged that this occurred from the error of the DTM (Digital Terrain Model) used for orthogonal correction. In addition to the causes suggested by this paper, additional studies should be conducted on factors that may affect the position accuracy. 본 논문은 국토관측위성용으로 제작된 정밀영상생성시스템의 위치정확도 분석에 대해서 보고한다. 국토관측위성의 발사 예정 시기는 2021년으로, 아직 발사되지 않아 국토관측위성과 유사한 사양을 가지는KOMPSAT-3A 위성영상을 활용하여 분석하였다. 본 논문에서는 한반도를 촬영한 30장의 영상을 이용하여 초기센서모델의 위치정확도, 정밀센서모델의 모델점 위치정확도, 검사점을 활용한 정밀센서모델의 위치정확도, 정밀정사영상의 위치정확도에 대한 측정을 수행하였다. 본 연구는 정확한 GCP 확보 시 2 pixel 이내의 RMSE를갖는 것을 목표 위치정확도로 한다. 그 결과, 검사점을 활용한 정밀센서모델의 위치정확도는 남한에서 약 1.85 pixel, 북한에서 약 2.04 pixel의 위치정확도를 갖는 것을 확인하였으며, 정밀정사영상의 위치정확도는 남한에서약 1.15 m, 북한에서 약 3.23 m의 위치정확도를 갖는 것을 확인 할 수 있었다. 전반적으로 남한의 정확도에 비해북한의 정확도가 낮은 것을 확인 할 수 있었으며, 이는 북한지역 영상의 GCP(Ground Control Point) 품질이 남한의 GCP 품질에 비해 좋지 않았기 때문에 측정된 정확도에 영향을 준 것으로 확인되었다. 또한, 특히 북한지역에서 정밀센서모델 대비 정밀정사영상의 정확도가 다소 떨어지는 것을 확인할 수 있었다. 이는 북한 영상의정사보정 시 사용한 DTM의 정확도가 남한에 비해 좋지 않아 발생한 것으로 판단하였다. 향후 본 연구진이 제시한 원인 외, 위치정확도에 영향을 줄 수 있는 요인들에 대한 추가적인 연구가 이루어져야 할 것이다.

무인기 소프트웨어에서 처리된 표정요소를 이용한 도화품질 예측기술 개발 및 비교분석

임평채,손종환,김태정 대한원격탐사학회 2019 大韓遠隔探査學會誌 Vol.35 No.6

현재 현업에서 사용되고 있는 상용 무인기 영상처리 소프트웨어는 카메라 캘리브레이션 정보나 영상전체에 대한 블록 번들조정 정확도만 제공할 뿐 스테레오 페어의 실제 도화 가능여부에 대한 정확도는 거의제공하지 않는다. 본 논문에서는 무인기 영상처리 소프트웨어에서 산출된 표정요소를 사용하여 도화품질을 산출하고 실제 도화기에 적용하여 도화품질의 신뢰성에 대해서 분석하였다. 도화품질은 Y시차 정확도, 상대모델 정확도, 절대모델 정확도의 3가지 정확도로 정의하였다. Y시차 정확도는 스테레오 페어간 입체시 여부를 판단할 수 있는 정확도이다. 상대모델 정확도는 모델 좌표계 상에서 스테레오 페어간 상대적인 번들조정정확도이다. 절대모델 정확도는 절대 좌표계에서 번들조정 정확도이다. 실험데이터는 도심지를 대상으로 회전익에서 취득된 GSD 5 cm급의 영상 723장을 사용하여 도화품질을 분석하였다. 연구진이 개발한 기술을 사용해 예측한 상대모델 정확도와 실제 도화기에서 관측한 정확도의 최대오차는 0.11 m로 정밀한 결과를 보여주었다. 절대모델 정확도도 마찬가지로, 도화기에서 관측한 정확도의 최대오차는 0.16 m로 정밀한 결과를 보여주었다. Commercial Unmanned Aerial Vehicle (UAV) image processing software products currently used in the industry provides camera calibration information and block bundle adjustment accuracy. However, they provide mapping accuracy achievable out of input UAV images. In this paper, the quality of mapping is calculated by using orientation parameters from UAV image processing software. We apply the orientation parameters to the digital photogrammetric workstation (DPW) for verifying the reliability of the mapping quality calculated. The quality of mapping accuracy was defined as three types of accuracy: Y-parallax, relative model and absolute model accuracy. The Y-parallax is an accuracy capable of determining stereo viewing between stereo pairs. The Relative model accuracy is the relative bundle adjustment accuracy between stereo pairs on the model coordinates system. The absolute model accuracy is the bundle adjustment accuracy on the absolute coordinate system. For the experimental data, we used 723 images of GSD 5 cm obtained from the rotary wing UAV over an urban area and analyzed the accuracy of mapping quality. The quality of the relative model accuracy predicted by the proposed technique and the maximum error observed from the DPW showed precise results with less than 0.11 m. Similarly, the maximum error of the absolute model accuracy predicted by the proposed technique was less than 0.16 m.

재무분석가의 특성이 이익예측정확성에 미치는 영향

안윤영 ( Yoon Young Ahn ),유영태 ( Young Tae Yoo ),조영준 ( Young Jun Cho ),신현한 ( Hyun Han Shin ),장진호 ( Jin Ho Chang ) 한국회계학회 2006 會計學硏究 Vol.31 No.4

본 연구는 IBES Detail Tape에 포함된 1999년부터 2003년까지의 재무분석가 이익예측자료를 사용하여, 재무분석가의 개인특성 변수를 중심으로 재무분석가의 이익예측정확성 결정요인에 대하여 살펴보았다. 분석결과 첫째, 많은 기업에 대한 분석을 담당하는 재무분석가일수록 이익예측오차는 높음을 발견하였다. 이는 재무분석가의 분석기업 수가 늘어날수록 분석업무의 복잡성이 증가하여 이익예측에 어려움을 겪고 있음을 의미한다. 둘째, 많은 분석보고서를 발행하는 재무분석가일수록 이익예측오차는 낮음을 발견하였다. 이는 특정기업에 대한 예측정보를 빈번하게 제공하여 활발한 분석활동을 수행하는 재무분석가일수록 이익예측능력이 높음을 의미한다. 셋째, 특정산업에 대한 분석을 실시하는 재무분석가일수록 이익예측오차가 낮음을 발견하였다. 이는 특정산업에 특화된 재무분석가일수록 분석업무의 복잡성이 감소하여 이익예측이 용이하였음을 의미한다. 마지막으로 기업특성 요인이 이익예측정확성에 미치는 영향을 분석한 결과 예측기간, 주가수익률 변동성, 표준화된 ROA, 부채비율 등이 높은 기업에서 이익예측오차가 높았으며, 재벌계열사에 소속된 기업에서 이익예측오차가 낮음을 발견하였다. Prior study finds that analysts` forecast activities(accuracy, coverage, the number of analysts following a firm, herding behavior, etc.) are related to several analysts characteristics(reputation, past accuracy, forecast frequency, firm-specific and general experience, and the number of firms and industries following) and environmental factors(employed brokerage size, and various characteristics of individual firms). Most of the empirical research on analysts` forecast accuracy in the current Korea security market focuses on environmental variables (individual company characteristics other than analyst characteristics. Therefore, our research has been conducted to extend the extant literatures` results on determinants of forecast accuracy using a sample of Korean firms included in the IBES database for a five year period from 1999 to 2003. We focus on the effect of analysts` characteristics on forecast accuracy. Since prior research provides mixed effects and results of analysts` characteristics on forecast accuracy, the direction of the effect of analysts` characteristics on forecast accuracy is unclear. We therefore established the non-directional hypothesis stated in the null form. To test whether each (analyst characteristics) variable has explanatory power incremental to the other, analyst characteristics variables are separately included in regression models respectively, and all variables are included as well. While the main focus of this study is to examine the effect of analyst characteristics on forecast accuracy, we have also considered the contribution of environmental variables (characteristics of individual firms) on forecast accuracy. The dependent variable of the basic research regression model is the forecast accuracy. Here forecast accuracy is defined as the logarithm of absolute forecast error and measured forecast error as the deviation of the actual EPS from the forecasted EPS which is deflated by the stock price to facilitate comparisons across firms. Thus more accurate forecasts are represented by lower forecast error values, i.e. there is an inverse relationship between forecast accuracy and forecast error. The three analyst characteristics (number of firms followed, forecast frequency and industry specialization) are key independent variables. In addition, we control for five variables of characteristics of firms (forecast horizon, stock price return volatility, standardized ROA, debt ratio and Chaebul dummy variable that prior research has shown to be related. First, we find that forecast accuracy decreases with the number of firms followed. The estimated coefficients of the number of firms followed are positively related to analysts` forecast error at the 1% significance level in all models represented by table 5 and table 6. This result indicates that the number of firms followed can be interpreted as a proxy for analysts` task (forecast) difficulty (portfolio complexity). If analysts follow a larger set of firms then it is difficult to devote more attention to each firm and to produce accurate forecasts. Second, forecast accuracy increases with the number of forecasts reports issued. The coefficients on the number of reports issued are negatively related to forecast error at the 5% or 1% significance level respectively presented by table5 and table 6. This result suggests that forecast error is significantly lower for analysts with more frequent reports. The number of forecasts reports issued can be considered as a proxy for analysts` skillful performance outputs and forecast ability. Third, forecast precision increases with the analyst industry specialization. This finding means that forecast error is significantly lower for analysts with a higher percentage of the companies followed by analysts that are in the same industry classification. Because industry specialization is expected to result in more accurate forecasts, the coefficients of analyst industry specialization are negatively associated with forecast error to a 1% or 5% significance level respectively presented by table 5 and table 6. Our overall conclusion is that forecast accuracy is associated with analyst-specific characteristics. We also examined the firm-specific characteristics effect on forecast accuracy. Regarding firm-specific characteristics variables, the coefficients on the forecast horizon, stock price return volatility, standardized ROA, leverage and Chaebul dummy are significantly related to forecast error. Consistent with prior studies, we find that environmental factors do influence analysts` forecast accuracy. Taken as a whole, analyst forecast accuracy is influenced by analyst-related properties as well as by environmental factors. This study makes several contributions. First, we contribute to the growing literature on analysts` forecast accuracy of the emerging market. Second, we suggest that analysts` characteristics may be helpful for predicting forecast accuracy. Future research might focus on investigating whether capital market participants should consider analyst-related properties in forming earnings expectations and evaluating analyst performance.

역세권 특성에 따른 수요예측의 정확도 분석 - 서울시 지하철을 중심으로 -

김동준,김덕녕,양재환,성현곤,이성모 한국교통연구원 2010 交通硏究 Vol.17 No.1

Recently, there have been constant criticisms over the low accuracy of the rail demand forecast. The accurate demand forecast is important as the rail project requires copious budget and time. However, the study on the accuracy of the rail demand forecast has not attracted researchers, whereas it is necessary to understand the factors which affect the accuracy. Especially, different station tends to show different accuracy in the same rail project, it is also necessary to study on this characteristic. The aim of this study is to analyze the effect on the microscopic characteristics of the station impact area to the accuracy of demand forecast. 142 subway stations in Seoul are analyzed to draw 10 characteristic indexes - density, land use mix, number of companies, number of labors, purpose of land use, number of bus lines, number of short-range bus lines, average number of bus lines of bus stop, w or w/o transfer - with categorizing the station impact area to the land use and the public transit. In addition, estimated demand is also analyzed. The accuracy of demand forecast in each station is used to estimate the ratio of risk. As a result, firstly, in terms of the land use, the demand forecast is more accurate with the higher density, land use mix, number of companies and labors which decreases the overestimation. At the same time, the station in the residential area with steady demand shows the higher accuracy. Secondly, in terms with the transit system, the demand forecast is more accurate when there are more bus lines connected to the subway station, on the other hand, the larger demand leads to the lower accuracy. Besides, the transit station shows lower accuracy than the non-transit station, which is supposed to be due to the limitation of the data. With the results above, it is discovered that the accuracy of rail demand forecast shows great differences between stations, which seems to be related with the characteristics of each station. Particularly, similar to the anticipated effect of TOD, an area with high density, mixed land use and feeder transit will improve the accuracy of rail demand forecast. 막대한 재원과 오랜 기간이 소요되는 철도사업의 수요예측 정확도가 낮아 비효율성을 초래한다는 지적이 지속적으로 제기되고 있다. 그러나 이와 관련된 연구는 많지 않은 실정으로, 수요예측의 정확도에 영향을 미치는 다양한 요인을 파악하고 분석하는 것이 필요하다. 특히 하나의 노선이라 할지라도 각 역별로 수요예측 정확도는 큰 차이가 발생하게 되는데, 역별 수요예측의 정확도를 높여 전체 노선의 정확도를 향상시키는 것이 중요하다. 본 연구에서는 서울시에 위치한 142개 지하철역을 대상으로 미시적 역세권 특성이 수요예측의 정확도에 미치는 영향을 파악하였다. 이를 위해 먼저 역세권 특성을 개발밀도, 토지이용의 다양성, 사업체수, 종사자수, 토지이용 목적 등의 토지이용 특성과 버스노선수, 환승/비환승역의 유무 등의 대중교통 특성으로 구분하고 GIS 등의 자료를 활용하여 파악하였다. 또한 지역별 수요예측의 정확도를 위험도라는 지표를 활용하여 산정하였다. 그 결과 개발밀도가 높을수록, 토지이용의 다양성이 높을수록, 사업체수 및 종사자수가 많을수록 수요예측의 정확도가 높아 과다 예측된 경우가 적은 것으로 나타났다. 고정적으로 통행이 발생하는 주거중심의 역이 상대적으로 정확도가 높은 것으로 나타났다. 또한 지하철과 연계되는 버스노선수가 많을수록, 연계성을 갖는 단거리 버스노선수가 많을수록 예측이 정확했으며, 환승역의 경우 비환승역에 비해 오히려 정확도가 낮게 나타났는데, 이는 자료의 한계로 인한 결과라 판단된다. 더불어 예측수요가 많을수록 정확도는 오히려 감소하는 것으로 나타났다. 본 연구결과를 통해 철도사업 수요예측의 정확도는 역별로 큰 차이를 보이며, 이는 역세권의 특성과 관련이 있음을 보여주고 있으며, 향후 철도사업 수요예측 정확도 향상을 위한 기초자료로 활용될 수 있을 것이다. 특히 TOD의 기대효과와 유사하게 개발밀도를 높이고 토지이용을 다양화하며, 연계교통수단의 서비스수준을 향상시킴으로써 수요예측의 정확도를 높일 수 있음을 보여주는 결과라 할 수 있다.

Re-examining Accuracy Measures in Task-based Research: Task Repetition

On-Soon Lee,Minyoung Cho 이화여자대학교 교과교육연구소 2021 교과교육학연구 Vol.25 No.5

As a language performance measure, accuracy has been considered a relatively consistent and unproblematic construct; however, accuracy can be assessed in a variety of ways, which may affect the results. This study examines how different measures of accuracy explain changes in task performance in the context of task repetition. Three different accuracy measures were explored: a rater’s holistic assessment, a frequency-based global measure of accuracy that counts all errors, and a specific measure of accuracy based on specific error types. Twenty-four EFL learners completed two picture narration tasks, repeating each task twice. They also completed a transcription activity between the two narration sessions, which functioned as an attention-drawing intervention and provided information on which specific error types to examine in the specific measure of accuracy. The learners’ picture narrations were analyzed, and the effects of task repetition examined, using each of the different accuracy measures. Findings showed different results by measures; specifically, improvement in accuracy was observed only through the holistic measure, while no evidence of improvement was found through the global and specific measures. These findings suggest that a holistic measure of accuracy offers a useful supplement to other measures for gauging the effectiveness of task-based language teaching.

The relationship among counselor experience level, empathic accuracy, and counseling outcome in the early phase of counseling

Kyung In Kwon,조수연 서울대학교 교육연구소 2012 Asia Pacific Education Review Vol.13 No.4

The current study examined a mediating effect of empathic accuracy on the relationship between counselor experience level and counseling outcome. Data were collected from 48 counselor-client dyads in real-life counseling settings. Empathic accuracy and counseling outcome were assessed by using client perceptions of the first three audiotaped counseling sessions. Ickes' standard empathic accuracy assessment procedure was used with modifications to assess empathic accuracy. The procedure to assess empathic evaluation included three steps: (a) clients recognize and write down their thoughts and feelings while listening to the audiotapes, (b) counselors infer their client's thoughts and feelings, (c) The third raters rate the statements of counselors' empathic accuracy. Correlational analysis revealed that counselor experience level, empathic accuracy, and counseling outcome were positively related to one another. A path analysis was used to test the model that counselor experience level affects counseling outcome through the effect of empathic accuracy. Both paths from counselor experience level to empathic accuracy and from empathic accuracy to counseling outcome were significant, along with a significant mediating effect of empathic accuracy. These results show enhancing empathic accuracy is critical for counseling outcome. Limitations of the current study and suggestions for future research were discussed.

Overview of the Process of Conducting Meta-analyses of the Diagnostic Test Accuracy

( Young Ho Lee ) 대한류마티스학회 2018 대한류마티스학회지 Vol.25 No.1

Diagnosis is a critical step for clinical treatment. Many individual studies have been conducted to determine the accuracy of various diagnostic tests, but they had small sample sizes and correspondingly inadequate statistical strength. Combining the results from several such studies can help increase the statistical strength and precision of their results. Meta-analysis is a useful tool for evaluating the accuracy of diagnostic tests and can be used to obtain precise estimates when multiple small studies for a given test and subject pool are available. The need for meta-analysis on studies examining diagnostic test accuracy has increased noticeably, and more meta-analyses on diagnostic test accuracy studies are being published. A meta-analysis of diagnostic test accuracy studies differs from a typical meta-analysis because diagnostic test accuracy studies report a pair of statistics, such as sensitivity and specificity, rather than a single statistic. Therefore, meta-analyses of the diagnostic test accuracy need to deal with two summary statistics simultaneously. More complex statistical methods are required for conducting meta-analyses using diagnostic test accuracy studies compared to that required for conventional meta-analysis. This is because the sensitivity and specificity are generally inversely correlated due to a threshold effect, and there is considerable heterogeneity in the results of test accuracy studies. This review provides an overview of the process of meta-analysis of the diagnostic test accuracy. (J Rheum Dis 2018;25:3-10)