결측치가 있는 준거참조검사의 일치도 계수

김진욱(Kim, Jinuk) 한국사회체육학회 2021 한국사회체육학회지 Vol.- No.84

Purpose: The purpose of this study is to analyze coefficients of agreement used as an evaluation tool for criterion-referenced tests. In common situations where more than three raters and more than three categories with missing data, we propose a method of easily calculating coefficients of agreement using vectors and matrices and modify the existing method of calculating the coefficients. Method: The data was summarized by raw data vectors, and by using them, two contingency tables for the subject by category and for rater by category were created. Results: The contingency tables were then used to calculate four frequently used coefficients of agreement, S (Benett et al., 1954), π (Scott, 1955), κ (Cohen, 1960), and AC1 (Gwet, 2002). Conclusion: By clearly defining different chance agreements assumed in each coefficient of agreement, an appropriate calculation could be made and a more valid coefficient of agreement could be obtained.

Smart Virtual Machine Placement Using Learning Automata to Reduce Power Consumption in Cloud Data Centers

Hossein Ghiasi,Mostafa Ghobaei Arani 한국산학기술학회 2015 SmartCR Vol.5 No.6

Today, cloud computing is one of the most challenging research topics in the field of information technology. It is so important for computer researchers that it was included on a list of top ten technologies in the world. Data centers include reservoirs where processing power can meet the needs of many users" computing. The popularity and acceptance of cloud computing has increased the number of these centers in recent years. One of the challenging issues in cloud computing environments is high energy consumption in data centers, which has been ignored in the corporate competition to develop data centers. High energy consumption by data centers leads to increased costs, as well as CO2 emissions. Researchers are now struggling to find an effective approach to decrease energy consumption in data centers. In recent years, many attempts have been made to reduce the power consumption of data centers, and many approaches have been proposed to reduce power consumption, such as hardware and software approaches and approaches using virtualization technology. In fact, placement of a virtual machine (VM) means finding a suitable physical place for the VM. The placement goal can either maximize the usage of available resources or it can save power by being able to shut down some servers. In this paper, we present an approach based on a best-fit decreasing (BFD) algorithm, which uses learning automata to reach a compromise between decreasing energy consumption and violating service level agreements.

자연과학편 : 전문가들의 주관적 평가에 대한 객관적 지수: 동의계수들의 비교

신진이(JhinYiShin),조은형(EunHyungCho),엄한주(HanJooEom) 한국체육학회 2015 한국체육학회지 Vol.54 No.6

체육 및 스포츠과학 분야의 다양한 영역에서 수행되고 있는 질적, 양적의 관찰연구에서 전문가들의 주관적 판단이 요구되는 경우가 매우 빈번하게 발생된다. 전문가들의 관찰대상은 사람일경우도 있고 사물의 특성(속성)을 분류, 평가하는 경우도 있다. 특히 학교현장에서의 수행능력 평가, 스포츠현장에서 요구되는 체력, 기술, 경기수행에 대한 평가 및 경기기술 자격시험, 경쟁스포츠에서의 심판판정 등에 대한 전문가들의 평가는 다양한 척도의 자료로 제공된다. 전문가들의 평가결과에 대한 객관성 및 신뢰성에 대한 문제는 심사자들 서로간의 상호 일치 정도 또는 심사대상의 상대적 능력 판별 정도에 따라 전자는 심사자들이 제공한 자료의 동의성(agreement), 후자는 일관성(consistency) 신뢰도로 평가된다. 본 논문에서는 신뢰계수와 동의계수의 동이성(同異性)에 대하여 설명하였고, 다양한 상황에서 발생되는 심사자들의 평가결과에 대한 상호 일치성 정도를 추정하는 동의계수들의 개념에 대하여 고찰하였다. 범주형 자료에 적용되는 단순동의계수와 심사자들의 ‘우연일치’ 평가결과로 인한 ‘과대추정’ 단점을 보완한 우연보정 동의계수, 그리고 서열척도에서 심사자간의 근접한 평가 범주를 고려한 부분동의계수, 마지막으로 동간-비율척도에 적용되는 우연보정 가중동의계수 및 ANOVA 분석으로 추정되는 ICC지수들에 대하여 심사자(r)의 수 그리고 분류항목(k)의 수를 구분하여 다양한 동의계수 산출방법들을 비교 설명하였으며 다양한 연구상황에 적합한 동의계수 선택기준을 제시하였다. Observational studies in the fields of sport science are frequently required to have experts to evaluate the characteristics and/or performances of subjects and equally likely to classify objects according to the criteria provided. The assessment of inter-rater reliability is a main procedure to demonstrate consistency and agreement between observational ratings among multiple raters. Inter-rater agreement is considered as one of reliability kinds. Consistency is the degree to which how well scores distinguish relative standings of subjects. On the other hand, agreement is the degree to which how close scores are in absolute values. Thus, when consistency among raters is high, then we can assume agreement will be high, but low consistency may not indicate a low agreement among raters. This paper is a tutorial and overview of methodological issues related to the assessment of inter-rater agreement with a focus on the selection of appropriate agreement index. Computational examples of agreement statistics include simple percent agreement, chance-corrected kappa-like index, Cohen’s weighted kappa, generalized kappa, and various intra-class correlations, and they are compared and discussed. Guidelines to select appropriate statistics and interpretations of ICCs are also provided.

On Assessing Inter-observer Agreement Independent of Variables' Measuring Units

엄용환 한국데이터정보과학회 2006 한국데이터정보과학회지 Vol.17 No.2

Investigators use either Euclidean distance or volume of a simplex defined composed of data points as agreement index to measure chance-corrected agreement among observers for multivariate interval data. The agreement coefficient proposed by Um(2004) is based on a volume of a simplex and does not depend on the variables' measuring units. We consider a comparison of Um(2004)'s agreement coefficient with others based on two unit-free distance measures, Pearson distance and Mahalanobis distance. Comparison among them is made using hypothetical data set.

On Assessing Inter-observer Agreement Independent of Variables' Measuring Units

Um, Yong-Hwan Korean Data and Information Science Society 2006 한국데이터정보과학회지 Vol.17 No.2

Investigators use either Euclidean distance or volume of a simplex defined composed of data points as agreement index to measure chance-corrected agreement among observers for multivariate interval data. The agreement coefficient proposed by Um(2004) is based on a volume of a simplex and does not depend on the variables' measuring units. We consider a comparison of Um(2004)'s agreement coefficient with others based on two unit-free distance measures, Pearson distance and Mahalanobis distance. Comparison among them is made using hypothetical data set.

국내 일사량 추정을 위한 Angstrom-Prescott계수의 평가

현신우 ( Shinwoo Hyun ),김광수 ( Kwang Soo Kim ) 한국농림기상학회 2016 한국농림기상학회지 Vol.18 No.4

농업 생태계의 생산성을 예측하기 위한 모델의 필수 입력변수인 일사량은 비교적 적은 수의 기상관측소에서만 관측되고 있어, 이들 관측값을 대신하기 위해 일사량을 추정하는 모델들이 사용되고 있다. 특히, 간단한 계수를 사용하여 일조시간을 이용하는 Angstrom-Prescott (AP) 모델이 일사량 추정을 위해 가장 널리 쓰이고 있다. 국내에서 보편적으로 적용가능한 AP모델의 계수값을 탐색하기 위해 국내 20개 기상관측소의 30년간의 일단 위 관측자료를 입력자료로 사용하여 경험적으로 얻어진 계수와 Frere and Popov(1979)가 제시한 계수(AP_Frere)를 이용한 일사량을 추정하고, 이들의 신뢰도를 분석하였다. 또한, 전역최적화 과정을 통해 시공간적으로 신뢰도가 높은 일사량을 얻을 수 있는 AP계수의 범위를 탐색 하였다. 분석을 위해 월별, 년도별, 지역별로 추정값과 측정값 사이의 일치도를 계산하였다. AP_Frere를 사용한 결과 작물 생산성 예측을 위한 조건에서 일치도가 높게 나타났다. AP_Frere를 사용하였을 때 전역최적화를 통해 추정한 AP계수(AP_max)를 사용하였을 경우 보다 일치도가 낮았으나 경험적으로 얻어진 계수(AP_Choi)보다는 일치도가 높은 일사량 추정이 가능하였다. 전역최적화를 통해 일사량 추정치의 신뢰도를 분석한 결과, 신뢰도가 높은 일사량을 얻을 수 있는 AP계수의 범위는 년도별로는 좁게 분포하였으나 월별, 지역별로는 넓게 분포하였다. 그 중에서도 변이가 작은 범위는 지역별 일치도가 월별 일치도보다 넓게 분포하였다. AP_Frere는 각각의 경우에 대해 일치도가 높고 변이가 작은 범위에 속해 국내 조건에서 AP_Frere를 적용할 경우, 신뢰도 높은 일사량 추정치를 얻을 수 있을 것으로 예상되었다. Models to estimate solar radiation have been used because solar radiation is measured at a smaller number of weather stations than other variables including temperature and rainfall. For example, solar radiation has been estimated using the Angstrom-Prescott (AP) model that depends on two coefficients obtained empirically at a specific site (AP_Choi) or for a climate zone (AP_Frere). The objective of this study was to identify the coefficients of the AP model for reliable estimation of solar radiation under a wide range of spatial and temporal conditions. A global optimization was performed for a range of AP coefficients to identify the values of AP_max that resulted in the greatest degree of agreement at each of 20 sites for a given month during 30 years. The degree of agreement was assessed using the value of Concordance Correlation Coefficient (CCC). When AP_Frere was used to estimate solar radiation, the values of CCC were relatively high for conditions under which crop growth simulation would be performed, e.g., at rural sites during summer. The statistics for AP_Frere were greater than those for AP_Choi although AP_Frere had the smaller statistics than AP_max did. The variation of CCC values was small over a wide range of AP coefficients when those statistics were summarized by site. AP_Frere was included in each range of AP coefficients that resulted in reasonable accuracy of solar radiation estimates by site, year, and month. These results suggested that AP_Frere would be useful to provide estimates of solar radiation as an input to crop models in Korea. Further studies would be merited to examine feasibility of using AP_Frere to obtain gridded estimates of solar radiation at a high spatial resolution under a complex terrain in Korea.

선형혼합모형에 의한 급내상관계수

김진욱 ( Kim Jinuk ) 한국체육학회 2020 한국체육학회지 Vol.59 No.6

본 연구의 목적은 규준지향검사(norm-referenced test)의 신뢰도 평가에 사용되는 급내상관계수 ICC(intraclass correlation coefficient)를 혼합모형으로 접근하여 신뢰도 분석체계를 재정립하는 것이다. 신뢰도를 일관성과 일치도로 나누어 서로 다른 혼합모형을 수립하였으며 분산성분의 추정은 제한가능도(restricted maximum likelihood) 방법을 이용하였다. Bartko(1976)의 수정된 자료와, Shrout와 Fleiss(1979)의 자료를 실제 분석에 사용하였다. 반복측정의 시점 또는 문항을 고정효과로 설정한 것이 일관성 모형(Model C), 임의효과로 설정한 것이 일치도 모형(Model A)으로 기존의 모형에 의한 신뢰도 분류보다는 직관적으로 이해하기 쉽다. 또한 분산분석 추정법보다 제한가능도 추정법이 다양한 장점을 가진다. The purpose of this study is to redefine the reliability analysis system of intraclass correlation coefficients(ICC), used in evaluating reliability of norm-referenced test(NRT), by approaching linear mixed models. Three types of model, Model 1(one-way random effects model), Model C(two-way mixed model, and Model A(two-way random effects model), one factor for subjects and the other factor for occasions, were developed and explained with respect to implications of mixed model. Using data from Bartko(1976) and Shrout and Fleiss(1979) variance components for ICC were estimated by restricted maximum likelihood(REML) and two kinds of reliability coefficient, consistency and agreement, were calculated. Linear mixed models are adequate and intuitive for ICC and REML estimation has many advantages over ANOVA estimation, which has been used traditionally.

Selection and Reporting of Statistical Methods to Assess Reliability of a Diagnostic Test: Conformity to Recommended Methods in a Peer-Reviewed Journal

Park, Ji Eun,Han, Kyunghwa,Sung, Yu Sub,Chung, Mi Sun,Koo, Hyun Jung,Yoon, Hee Mang,Choi, Young Jun,Lee, Seung Soo,Kim, Kyung Won,Shin, Youngbin,An, Suah,Cho, Hyo-Min,Park, Seong Ho unknown 2017 KOREAN JOURNAL OF RADIOLOGY Vol.18 No.6

<P><B>Objective</B></P><P>To evaluate the frequency and adequacy of statistical analyses in a general radiology journal when reporting a reliability analysis for a diagnostic test.</P><P><B>Materials and Methods</B></P><P>Sixty-three studies of diagnostic test accuracy (DTA) and 36 studies reporting reliability analyses published in the <I>Korean Journal of Radiology</I> between 2012 and 2016 were analyzed. Studies were judged using the methodological guidelines of the Radiological Society of North America-Quantitative Imaging Biomarkers Alliance (RSNA-QIBA), and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. DTA studies were evaluated by nine editorial board members of the journal. Reliability studies were evaluated by study reviewers experienced with reliability analysis.</P><P><B>Results</B></P><P>Thirty-one (49.2%) of the 63 DTA studies did not include a reliability analysis when deemed necessary. Among the 36 reliability studies, proper statistical methods were used in all (5/5) studies dealing with dichotomous/nominal data, 46.7% (7/15) of studies dealing with ordinal data, and 95.2% (20/21) of studies dealing with continuous data. Statistical methods were described in sufficient detail regarding weighted kappa in 28.6% (2/7) of studies and regarding the model and assumptions of intraclass correlation coefficient in 35.3% (6/17) and 29.4% (5/17) of studies, respectively. Reliability parameters were used as if they were agreement parameters in 23.1% (3/13) of studies. Reproducibility and repeatability were used incorrectly in 20% (3/15) of studies.</P><P><B>Conclusion</B></P><P>Greater attention to the importance of reporting reliability, thorough description of the related statistical methods, efforts not to neglect agreement parameters, and better use of relevant terminology is necessary.</P>

초등학생(고학년)의 에너지필요추정량 산출을 위한 4단계 신체활동단계의 간편 판정법 연구

김미정(Mi Jeong Kim),나현주(Hyeon Ju Na),김영남(Young Nam Kim) 대한지역사회영양학회 2011 대한지역사회영양학회지 Vol.16 No.3

The purpose of this study is finding the easy way of 4 categories activity level confirmation for estimated energy requirement calculation. Total of 386, 5th and 6th grade primary school students participated. The time spent on 7 kinds of activity were collected for 1 day by the internet program developed. Judged by the activity coefficient, sedentary were 6.7% and 5.1%, low active 33.2% and 40.4%, active 43.8%, and45.5%, and very active 16.3% and 9.0% for boy and girl, respectively. The highest and significant correlation coefficient between activity coefficient and time spent on activities shown were 0.339 in commute activity for boys, and 0.466 in leisure for girls. The sensitivity of the sedentary conformation by commute hour for boys was 0.79, and that of very active was 0.56. The sensitivity of the sedentary conformation by leisure hour for girls was 0.67, and that of very active was 0.63. The sensitivity of low active and active by 7 different types of activity was quite low, 0.04~0.37. The exact agreement of activity level conformed by easy way developed was 30.8% and 33.7%, for boys and girls, respectively. More accurate way to identify 4 categories activity level needs to be developed, especially sensitive to conformation of low active and active levels. (Korean J Community Nutr 16(3) : 307~314, 2011)

Selection and Reporting of Statistical Methods to Assess Reliability of a Diagnostic Test: Conformity to Recommended Methods in a Peer-Reviewed Journal

박지은,한경화,성유섭,정미선,구현정,윤희망,최영준,이승수,김경원,신영빈,안수아,조효민,박성호 대한영상의학회 2017 Korean Journal of Radiology Vol.18 No.6

Objective: To evaluate the frequency and adequacy of statistical analyses in a general radiology journal when reporting a reliability analysis for a diagnostic test. Materials and Methods: Sixty-three studies of diagnostic test accuracy (DTA) and 36 studies reporting reliability analyses published in the Korean Journal of Radiology between 2012 and 2016 were analyzed. Studies were judged using the methodological guidelines of the Radiological Society of North America-Quantitative Imaging Biomarkers Alliance (RSNA-QIBA), and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. DTA studies were evaluated by nine editorial board members of the journal. Reliability studies were evaluated by study reviewers experienced with reliability analysis. Results: Thirty-one (49.2%) of the 63 DTA studies did not include a reliability analysis when deemed necessary. Among the 36 reliability studies, proper statistical methods were used in all (5/5) studies dealing with dichotomous/nominal data, 46.7% (7/15) of studies dealing with ordinal data, and 95.2% (20/21) of studies dealing with continuous data. Statistical methods were described in sufficient detail regarding weighted kappa in 28.6% (2/7) of studies and regarding the model and assumptions of intraclass correlation coefficient in 35.3% (6/17) and 29.4% (5/17) of studies, respectively. Reliability parameters were used as if they were agreement parameters in 23.1% (3/13) of studies. Reproducibility and repeatability were used incorrectly in 20% (3/15) of studies. Conclusion: Greater attention to the importance of reporting reliability, thorough description of the related statistical methods, efforts not to neglect agreement parameters, and better use of relevant terminology is necessary.