A Study on Evaluation Items for the Sustainability of Development Projects : Focused on KOICA’s End-of-Project Evaluation

다국어 초록 (Multilingual Abstract)

Achieving and sustaining the intended outcomes of development projects is the ultimate goal of international development cooperation. However, despite its significance, research on the sustainability of development projects and its evaluation has been insufficient. In particular, sustainability criteria have not been adequately addressed in the evaluation of Korean ODA projects, and studies aimed at improving this aspect remain limited. Given these concerns, this study aims to establish a more systematic approach to sustainability evaluation. To achieve this, a comprehensive review of previous studies was conducted to identify factors influencing the sustainability of development projects, and a framework for the influential factors of sustainability was established based on these findings. Subsequently, a Delphi study was conducted to derive 21 core sustainability evaluation items appropriate for end-of-project evaluations, and their relative importance was analyzed using the Analytic Hierarchy Process (AHP). Furthermore, a meta-evaluation using content analysis was performed on 134 KOICA end-of-project evaluation reports to examine the current state of sustainability evaluations. The analysis revealed that KOICA’s end-of-project evaluations primarily focus on recipient countries' financial, human, and policy resources, while other critical factors tend to be overlooked. Additionally, sustainability evaluation items were not comprehensively applied, and only a limited number of items were explicitly reflected in the evaluations. In-depth interviews further revealed that the low quality of sustainability evaluations is largely attributable to factors such as evaluators' lack of expertise and the absence of detailed evaluation guidelines. This study contributes to development evaluation research by quantitatively validating sustainability evaluation items and diagnosing the current state of sustainability evaluations. Methodologically, it enhances the rigor of sustainability evaluation research by employing expert consensus methods and systematic content analysis. Practically, the findings provide valuable implications for revising KOICA’s evaluation guidelines, strengthening evaluator capacity, and improving the overall sustainability evaluation framework. However, this study primarily focuses on prospective sustainability within KOICA’s bilateral project end-of-project evaluations and does not encompass the long-term continuation of project outcomes (actual sustainability). Future research should expand the scope of sustainability evaluation studies to include various project types and sectors while verifying the applicability of the proposed sustainability evaluation items in actual evaluations. Furthermore, this study offers insights applicable beyond the field of evaluation to the broader domain of development cooperation. Projects that do not thoroughly consider sustainability factors during planning are inherently less likely to be sustainable, and sustainability constraints may arise if recipient institutions are not adequately prepared to maintain project outcomes and key activities during project implementation process. The establishment of more comprehensive sustainability evaluation items, as proposed in this study, provides guidance for stakeholders involved in project planning and implementation, helping them recognize critical sustainability factors in their decision-making processes.

목차 (Table of Contents)

• Ⅰ. Introduction 1
• 1. Background 1
• 1.1. Sustainability and Its Evaluation Challenges in Korea’s ODA 1
• 1.2. Research Focus and Identified Gaps 3
• 2. Research Design 6
• 2.1. Research Questions 6
• 2.2. Research Process 8
• 3. Structure of the Thesis 9
• Ⅱ. Theoretical Background 10
• 1. Concept of Sustainability 10
• 1.1. Definition of Sustainability 10
• 1.2. Operational Definition 12
• 2. Evaluation of Project Sustainability 13
• 2.1. OECD DAC Evaluation Criteria 14
• 2.2. Analysis of Evaluation Guidelines from Development Agencies 15
• 3. Influential Factors of Project Sustainability 20
• 3.1. Existing Frameworks for Influential Factors of Sustainability 20
• 3.2. Proposed Framework for Influential Factors of Sustainability 30
• 3.3. Influential Factors of Sustainability by Domain 38
• 4. Chapter Conclusion 52
• Ⅲ. Validation of Core Sustainability Evaluation Items 54
• 1. Study Methodology 54
• 1.1. Overview of the Delphi Technique 54
• 1.2. Principles of Conducting and Reporting Delphi Study 55
• 2. Delphi Study Design 56
• 2.1. Rationale for Using the Delphi Technique 56
• 2.2. Delphi Study Process 57
• 3. Results of the Delphi Study 63
• 3.1. Round 1 Results 63
• 3.2. Round 2 Results 70
• 4. Chapter Conclusion 77
• Ⅳ. Prioritization of Sustainability Evaluation Items 78
• 1. Study Methodology 78
• 1.1. Rationale for Applying the AHP Method 78
• 1.2. AHP Research Process 79
• 2. AHP Design 81
• 2.1. Selection of Respondents 81
• 2.2. Hierarchy Construction 82
• 2.3. Questionnaire Design 83
• 2.4. Consistency Criteria 85
• 3. AHP Results 85
• 3.1. Data Collection 85
• 3.2. Respondent Demographics 86
• 3.3. AHP Analysis Results 87
• 4. Chapter Conclusion 99
• Ⅴ. Meta-Evaluation of KOICA End-of-Project Evaluations 101
• 1. Study Methodology 101
• 1.1. Meta-Evaluation 101
• 1.2. Content Analysis 103
• 2. Study Design 106
• 2.1. Data and Sampling 106
• 2.2. Data Coding 107
• 2.3. Reliability Test 108
• 3. Analysis Results 110
• 3.1. Descriptive Statistics of Sampled Evaluation Reports 110
• 3.2. Analysis of the Application of Sustainability Evaluation Items 113
• 3.3. Underlying Causes of the Discrepancies 121
• 4. Chapter Conclusion 126
• Ⅵ. Conclusion 128
• 1. Key Findings 128
• 2. Discussion and Implication 130
• 3. Contribution, Limitations, and Areas for Further Research 137
• 3.1. Contribution 137
• 3.2. Limitations 138
• 3.3. Areas for Further Research 139
• References 141
• Appendix 152