Overview of the Process of Conducting Meta-analyses of the Diagnostic Test Accuracy

( Young Ho Lee ) 대한류마티스학회 2018 대한류마티스학회지 Vol.25 No.1

Diagnosis is a critical step for clinical treatment. Many individual studies have been conducted to determine the accuracy of various diagnostic tests, but they had small sample sizes and correspondingly inadequate statistical strength. Combining the results from several such studies can help increase the statistical strength and precision of their results. Meta-analysis is a useful tool for evaluating the accuracy of diagnostic tests and can be used to obtain precise estimates when multiple small studies for a given test and subject pool are available. The need for meta-analysis on studies examining diagnostic test accuracy has increased noticeably, and more meta-analyses on diagnostic test accuracy studies are being published. A meta-analysis of diagnostic test accuracy studies differs from a typical meta-analysis because diagnostic test accuracy studies report a pair of statistics, such as sensitivity and specificity, rather than a single statistic. Therefore, meta-analyses of the diagnostic test accuracy need to deal with two summary statistics simultaneously. More complex statistical methods are required for conducting meta-analyses using diagnostic test accuracy studies compared to that required for conventional meta-analysis. This is because the sensitivity and specificity are generally inversely correlated due to a threshold effect, and there is considerable heterogeneity in the results of test accuracy studies. This review provides an overview of the process of meta-analysis of the diagnostic test accuracy. (J Rheum Dis 2018;25:3-10)

Meta-Analysis of Diagnostic Test Accuracy

이영호 한양대학교 의과대학 2015 Hanyang Medical Reviews Vol.35 No.1

Diagnosis is a critical step in clinical treatment. Meta-analysis is a useful tool for evaluating the accuracy of diagnostic tests and can be used to obtain precise accuracy estimates when small studies for a given test and subject pool are available. Meta-analysis uses statistical techniques to combine and compare data from different studies, thus increasing the power of the estimates of diagnostic accuracy in primary research. Meta-analysis of diagnostic tests summarizes the accuracy of diagnosis. Therefore, it is necessary for clinicians to understand meta-analytical procedures for diagnostic tests. Herein, we describe the basic steps in a meta-analysis to evaluate test accuracy: 1) describing the results of individual studies, 2) searching for heterogeneity, 3) testing for the threshold effect, 4) deciding on the model for statistical pooling, 5) dealing with heterogeneity, and 6) interpreting meta-analysis results of diagnostic tests. Meta-analysis of diagnostic test accuracy provides useful information for clinical practice and for the formulation of questions to be tested in future studies.

Diagnostic Accuracy of Clinical Test for Anterior Cruciate Ligament Injury: Systematic Review

데니즈 야스민,변상우,이형수,김성은,한선규,이동엽,홍지헌,유재호,김진섭,김성길 대한물리치료학회 2023 대한물리치료학회지 Vol.35 No.3

Purpose: The aim of this study was to conduct a systematic review of randomized controlled studies from 2012 to present that explore the diagnostic accuracy of clinical tests used for diagnosing anterior cruciate ligament (ACL) injury. Methods: Study design: Systematic review. Literature search of the PubMed and Scholar databases was conducted using keywords re- lated to diagnostic accuracy of clinical tests for ACL injury. The PRISMA Guidelines were followed to conduct this study. The Cochrane Risk of Bias Tool was utilized to assess the quality of each included study. Results: As a result, 8 studies were included, and 6 clinical tests used in ACL tears were evaluated for diagnostic accuracy. The pivot shift test was reported as having the highest +LR (29.5) value with a sensitivity of 59% and a specificity of 98%. However, the test with the lowest -LR value was the lever test, and the values were as follows: -LR (0.08), +LR (4.7), specificity (80%), sensitivity (94%). Conclusion: In this study, it was concluded that a single clinical test is not sufficient to determine the presence of ACL injury. Test com- binations have a higher diagnostic accuracy than a single test. In this study, the accuracy of the clinical tests was examined without considering the amount of ACL rupture and acute-chronic condition. Further research is required to examine the impact of these two factors on diagnostic accuracy of clinical test.

Diagnostic Accuracy of the Initial Endoscopy for Ampullary Tumors

이희승,장종순,이승호,연명호,김기배,박재근,이주영,김미진,한정호,성노현,박선미 대한소화기내시경학회 2015 Clinical Endoscopy Vol.48 No.3

Background/Aims: Ampullary tumors come in a wide variety of malignant forms. We evaluated the diagnostic accuracy of endoscopy for ampullary tumors, and analyzed the causes of misdiagnosis. Methods: We compared endoscopic imaging and biopsy results to final diagnoses. Types of endoscope, numbers of biopsy specimens taken, and final diagnoses were evaluated as possible factors influencing diagnostic accuracy. Results: Final diagnoses were 19 adenocarcinomas, 18 normal or papillitis, 11 adenomas, two adenomyomas, one paraganglioma, and one neuroendocrine tumor. The diagnostic accuracy of endoscopic imaging or the initial biopsy was identical (67.3%). At least one test was concordant with the final diagnosis in all except two cases. Compared with the final diagnosis, endoscopic imaging tended to show more advanced tumors, whereas the initial biopsy revealed less advanced lesions. The diagnostic accuracy of the initial biopsy was influenced by the type of endoscope used and the final diagnosis, but not by the number of biopsies taken. Conclusions: Endoscopy has limited accuracy in the diagnosis of ampullary tumors. However, most cases with concordant endoscopic imaging and biopsy results are identical to the final diagnosis. Therefore, in cases where both of these tests disagree, re-evaluation with a side-viewing endoscope after resolution of papillitis is required.

Systematic Review and Meta-Analysis of Studies Evaluating Diagnostic Test Accuracy: A Practical Review for Clinical Researchers–Part II. Statistical Methods of Meta-Analysis

이준영,김경원,최상현,허지미,박성호 대한영상의학회 2015 Korean Journal of Radiology Vol.16 No.6

Meta-analysis of diagnostic test accuracy studies differs from the usual meta-analysis of therapeutic/interventional studies in that, it is required to simultaneously analyze a pair of two outcome measures such as sensitivity and specificity, instead of a single outcome. Since sensitivity and specificity are generally inversely correlated and could be affected by a threshold effect, more sophisticated statistical methods are required for the meta-analysis of diagnostic test accuracy. Hierarchical models including the bivariate model and the hierarchical summary receiver operating characteristic model are increasingly being accepted as standard methods for meta-analysis of diagnostic test accuracy studies. We provide a conceptual review of statistical methods currently used and recommended for meta-analysis of diagnostic test accuracy studies. This article could serve as a methodological reference for those who perform systematic review and meta-analysis of diagnostic test accuracy studies.

A Meta-analysis of the Diagnostic Value of Minor Salivary Gland Biopsy for Primary Sjogren`s Syndrome

( Young Ho Lee ),( Gwan Gyu Song ) 대한류마티스학회 2014 대한류마티스학회지 Vol.21 No.6

Objective. The purpose of this study is to evaluate the diagnostic performance of minor salivary gland biopsy (MSGB) for patients with primary Sjogren’s syndrome (pSS).Methods. We have conducted a search from Medline, Embase, and Cochrane Library databases, and performed a meta-analysis on the diagnostic accuracy of MSGB in pSS patients. Results. A total of eight studies, including 583 pSS and 627 non-pSS patients, were available for the meta-analysis. The pooled sensitivity and specificity of MSGB were 75.7% (95% confidence interval [CI], 72.0∼79.1) and 90.7% (88.1∼92.9), respectively. The positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio were 9.475 (4.051∼22.16), 0.266 (0.208∼0.340), and 38.92 (19.12∼72.21), respectively. The area under the curve was 0.901 and the Q* index was 0.902, indicating a high diagnostic accuracy. Some between-study heterogeneity was found in the meta-analyses; however, there was no evidence of a threshold effect (Spearman correlation coefficient=0.419; p=0.301). Meta-regression showed that the study quality, sample size, study design, and diagnostic criteria were not sources of heterogeneity, and subgroup meta-analyses did not change the overall diagnostic accuracy.Conclusion. Our meta-analysis of published studies demonstrates that MSGB has a high diagnostic accuracy and may play an important role in the diagnosis of pSS.

Diagnostic test accuracy: application and practice using R software

Sung Ryul Shim,Seong-Jang Kim,Jonghoo Lee 한국역학회 2019 Epidemiology and Health Vol.41 No.-

The objective of this paper is to describe general approaches of diagnostic test accuracy (DTA) that are available for the quantitative synthesis of data using R software. We conduct a DTA that summarizes statistics for univariate analysis and bivariate analysis. The package commands of R software were “metaprop” and “metabin” for sensitivity, specificity, and diagnostic odds ratio; forest for forest plot; reitsma of “mada” for a summarized receiver-operating characteristic (ROC) curve; and “metareg” for meta-regression analysis. The estimated total effect sizes, test for heterogeneity and moderator effect, and a summarized ROC curve are reported using R software. In particular, we focus on how to calculate the effect sizes of target studies in DTA. This study focuses on the practical methods of DTA rather than theoretical concepts for researchers whose fields of study were non-statistics related. By performing this study, we hope that many researchers will use R software to determine the DTA more easily, and that there will be greater interest in related research.

의료보험·보호 자료에 표기된 정신과 진단의 진단 정확도 : 경기도 연천지역 자료에서

황성혜,조맹제,홍진표 大韓神經精神醫學會 1996 신경정신의학 Vol.35 No.6

본 연구는 의료보험·보호 자료상에 표기된 정신과적 진단의 진단 정확도를 알아봄으로써 그 자료를 향후의 정신보건 연구나 정책설정 과정에 유용하게 사용할 수 있는 방안을 모색하는 것을 목적으로 하였다. 경기도 연천군에서 1993년 한 해 동안 의료보험공단에 청구된 의료기관 이용자 중 정신과 진단(ICD-9-290-319)을 받은 사람들 중에서 346명을 대상으로 하였다. 연구자들이 제작한 반구조화된 면담도구를 사용하여 실시하였다. 그 면담을 통하여 얻은 자료를 통하여 연구자들이 ICD-9에 기초한 일치 진단을 얻었다. 연구자들의 진단과 보험자료상 가장 자주 표기된 진단을 비교하였다. 결과는 다음과 같다. 정신분열병성 정신증(Schizophrenic psychoses ; ICD-9 295)은 예민도 0.77 특이도 0.97 kappa치 0.77, 정동성 정신증(Affective psychoses ; ICD-9 296)은 예민도 0.28 특이도 0.94 kappa치 0.27, 신경증적 장애(Neurotic disorders ; ICD-9 300)는 예민도 0.78 특이도 0.73 kappa치 0.40, 알콜의존 증후군(Alcohol dependence syndrome ; ICD-0 303)은 예민도 0.46 특이도 0.99 kappa치 0.57, 다른 곳에 분류되지 않은 특수 증상 혹은 증후군(Special symptoms or syndromes not elsewhere classified ; ICD-9 307)는 예민도 0.47 특이도 0.94 kappa 0.39였고 전체 kappa치는 0.46이었다. 결론적으로, 전체 kappa치는 0.46으로 비교적 수용할 수 있는 수준이나 특히 정동성 정신증(Affective psychoses)의 진단에 있어서 진단의 정확도가 떨어졌다. 그 원인으로는 비정신과의사의 우울장애 인지율 저조, ICD-9 진단분류상의 모호함 등을 들 수 있다. 그 개선방안으로는 일차진료를 맡게 되는 비정신과의사들에게 정신보건에서 중요한 질환과 분류체계에 대한 교육, 특히 우울증에 관한 교육이 필요하며 간단한 자기보고형의 선별검사들이 우울증의 인지에 도움이 될 것으로 생각된다. Objective : To evaluate the diagnostic accuracy of the psychiatric diagnoses described in medical security records. Method : Using medical security records in Yonchon in 1993, 346 patients with psychiatric diagnoses(ICD-9 290-319) were sampled for the study. 13 trained interviewers visited the patients' home and interviewed with semistructured interview schedule made by the authors. 13 interviewers consisted of public health doctors, social worker, nurses, psychiatric residents and psychiatrist. On each interview data of the sample patients, consensus diagnoses were made by the authors according to ICD-9 and DSM-Ⅲ-R criteria. Result : Sensitivity, specificity and kappa value of Schizophrenic psychoses(ICD-9 295) were 0.77, 0.97 and 0.77, respectively. Those of Affective psychoses(ICD-9 296) were 0.28, 0.94 and 0.27, respectively. Those of Neurotic disorders(ICD-9 300) were 0.78, 0.73 and 0.40, respectively. Those of Alcohol dependence syndrome(ICD-9 303) were 0.46, 0.99 and 0.57, respectively. Those of Special symptoms and syndromes not elsewhere classified(ICD-9 307) were 0.47, 0.94 and 0.39, respectively. Overall kappa value was 0.46. Conclusion : Overall kappa value was moderately acceptable. But the diagnostic accuracy of Affective psychoses was very low. It may be due to low recognition rate of depression by primary care physicians, and the obscurity of ICD-9 classification for mood disorders. Psychiatric education for primary care physicians and using self-report questionnaire for screening test will be needed to improve the diagnostic accuracy.

The Orebro Musculoskeletal Pain Screening Questionnaire and the StarT Back Screening Tool: A systematic review and meta-analysis of Diagnostic test accuracy

Kyounghae Kim,Jinkyung Park 한국간호과학회 2021 한국간호과학회 학술대회 Vol.2021 No.10

Aim(s): Due to the physical, psychosocial, and economic consequences of chronic low back pain, it is essential to identify individuals with low back pain at an increased risk of the transition to chronicity and worse functional outcomes. The Orebro Musculoskeletal Pain Screening Questionnaire (OMPSQ) and the StarT Back Screening Tool (SBST) are commonly used screening tools. Yet, little is known about their diagnostic test accuracy. The aim is to evaluate the diagnostic test accuracy (DTA) of the OMPSQ and the SBST through systematic review and meta-analysis. Method(s): We searched the databases of EMBASE, PubMed, and Web of Science in June 2021. Study selection, extraction, and quality assessment using the Quality Assessment of the Diagnostic Accuracy Studies-2 tool were undertaken by two authors independently. We estimated the pooled DTA values by employing a bivariate model and hierarchical summary receiver-operating characteristic curves for data synthesis. Result(s): Of 3,044 records, we identified 17 studies, including 10 employing the OMPSQ and seven using the SBST. The pooled estimates of sensitivity and specificity of the OMPSQ scale for predicting sick leave were 85.1% (95% confidence interval [CI] 73.6%-92.1%) and 61.9% (95% CI 46.9%-74.9%) at cut-off 90, and 68.4% (95% CI 56.7%-78.2%) and 70.5% (95% CI 56.9%-81.2%) at cut-off 105, respectively. The pooled estimates of sensitivity and specificity of the SBST for predicting disability were 53.5% (95% CI 44.7%-62.1%) and 62.8% (95% CI 41.6%-80.0%) at Low vs Medium and High group, and 81.9% (95% CI 69.9%-89.9%) and 26.9% (95% CI 7.9%-61.1%) at Low and Medium vs High group, respectively. Conclusion(s): With the best evidence of the DTA of the SBST and OMPSQ, the OMPSQ is a reliable and useful tool to predict sick leave from low back pain and the SBST is useful for predicting disability due to low back pain.

인지진단 컴퓨터화 적응 검사에서 모수적 및 비모수적 문항선택 방법에 따른 분류 정확성 비교

한정아(Junga Han),이규민(Guemin Lee) 한국교육평가학회 2020 교육평가연구 Vol.33 No.1

최근 교육평가 분야에서 검사의 효율성과 추정의 정확성을 향상시키고 피험자에 대한 진단정보를 제공하기 위해 인지진단이론에 컴퓨터화 적응 검사를 적용한 인지진단 컴퓨터화 적응 검사가 주목받고 있다. 본 연구에서는 인지진단 컴퓨터화 적응 검사 상황에서 모수적, 비모수적 문항선택 방법에 따른 분류 정확성을 다양한 검사 조건(인지요소의 수, Q행렬의 구성, 문항의 진단성, 문제은행 구축 시 피험자의 수)에서 비교하고자 하였다. 인지진단 컴퓨터화 적응 검사의 문제은행은 DINA 모형을 적용하여 구축하였다. 연구 결과, 인지요소의 수가 많을수록, 문항의 진단성이 낮을수록 모수적 및 비모수적 문항선택방법 모두 분류 정확성이 감소하였다. Q행렬의 구성이 복잡할수록 모수적 문항선택 방법에서 분류 정확성이 감소하였으나, 비모수적 문항선택 방법에서는 Q행렬의 구성에 따른 분류 정확성 차이가 크게 나타나지 않았다. 인지진단 컴퓨터화 적응 검사를 시행하기 전문제은행 구축 단계에서 많은 인원을 확보할 수 없어 적은 인원으로 문항 모수를 추정하는 경우에는 비모수적 문항선택 방법을 적용할 때 분류 정확성을 더 높일 수 있다. Cognitive diagnostic computerized adaptive testing (CD-CAT) has been focused and investigated recently in the field of educational measurement. The CD-CAT applies the similar schemes of computerized adaptive testing to the context of cognitive diagnosis theory for improving testing efficiency and estimation accuracy. The purpose of this study were to compare parametric and nonparametric item selection methods in classification accuracy under various test conditions of CD-CAT, and to identify the effects of several factors such as number of attributes, Q-matrix composition, item diagnosticity, number of examinees for estimating item parameter. The DINA model was chosen as a basic model for parameter estimation in constructing item bank. The main conclusions can be made as follows. The greater the number of attributes and the lower the item diagnosticity, the lower the classification accuracy. As the Q-matrix composition become more complex, the classification accuracy decreased in the parametric item selection methods. But this conclusion can not be applied to the nonparametric item selection method. The examinee’s attribute pattern or mastery status were more accurately estimated in the nonparametric item selection method, when estimating item parameters with a small number of examinees.