대학생 글쓰기 기초능력 진단평가 방안 연구 - 사례 분석을 통한 진단평가의 개선 방안 고찰 -

조남민 이화어문학회 2019 이화어문논집 Vol.49 No.-

This research clarifies why diagnostic assessment, is required given the situation around writing education at college level, and how we could raise validity and reliability in the process of development and implementation of diagnostic assessment. This research also discusses how diagnostic assessment can be practically implemented in real life cases. To improve the effectiveness of writing education and diagnostic assessment, this research explores a way to develop a system to assess the basic writing abilities of college students and a way to implement it, with a follow-up process for initial diagnostic assessment. By utilizing diagnostic assessment, teachers can objectively assess the writing abilities of their students, and, based on this, they can provide customized writing courses for the top 10% and bottom 10% of students. Also, by applying the equivalent test to preliminary evaluation and post-evaluation the instructors and their students can assess improvement throughout the course. The students can recognize in detail where they have made improvements and where they have not among different components of writing abilities. Outside of classes, based on the analysis of diagnostic assessment’s preliminary evaluation and post-evaluation, teachers can improve writing education at college level in terms of the course structure, key contents, and teaching strategy. As a result of this cycle, the teachers can continuously enhance the effectiveness of writing education, and can adapt to various situations identified by diagnostic assessment. 본고는 현재 우리나라의 대학 글쓰기 교육이 처한 여러 여건을 고려했을 때 진단평가의 필요성이 제기된다는 점을 밝히고, 진단평가의 개발 절차와 시행에 있어서 타당도와 신뢰도를 높이기 위해 고려해야 할 요소를 제시하였다. 그리고 진단평가가 실제로 어떤 방식으로 실행되는지 사례를 통해 논의하였다. 더불어 대학생의 글쓰기 기초능력을 진단할 수 있는 평가 도구를 개발하는 절차와 이것의 실행 방안, 그리고 진단평가 시행 후 후속되어야 할 구체적인 교육 방안을 모색함으로써 진단평가의 효용성과 글쓰기 교육의 효율성을 높일 수 있음을 논의하였다. 진단평가를 통해 글쓰기 교과목을 수강하는 학생들의 글쓰기 실력을 객관화할 수 있으며, 이를 바탕으로 상위 10% 학생과 하위 10% 학생에게 수준별 맞춤형 글쓰기 교육을 진행할 수 있다. 더불어 사전 평가와 사후 평가를 동형검사로 진행함으로써 학생들은 글쓰기 학업 성취도를 구체적으로 알 수 있게 된다. 또한, 학생들은 글쓰기 능력을 구성하는 여러 항목 중 향상된 부분과 그렇지 못한 부분에 대해 인식할 수 있다. 수업 외적으로는 진단평가의 사전․사후 평가의 결과 분석을 통해 대학 글쓰기 교과목의 교육과정 및 교육 내용, 그리고 교수 전략을 발전적으로 수정할 수 있는 근거를 마련할 수 있으며, 수정․변경해야 할 영역에 대해 객관적인 판단을 할 수 있다. 궁극적으로 교과목 담당자는 글쓰기 교과목을 [사전 진단평가] → [수준별 학습] → [사후 진단평가] → [진단평가 결과 분석] → [교과 내용 수정/강화] → [다음 학기 수업 반영] 등의 과정에서 발전적으로 변화시킴으로써 여러 변화 요인에 대처할 수 있게 된다.

국어과 교과학습 진단평가 도구 개발 연구

박정진 한국언어문화교육학회 2008 언어와 문화 Vol.4 No.3

The purpose of this study is to develop the standardized instrument for diagnostic assessment of coursework in Korean Language Education. For this purpose, we develop the elements which can show the Korean Language Ability. We presented the elements of the Korean Language Ability that is related the 7th curriculum. The developed instrument was applied as a diagnostic assessment at two elementary schools in seoul. The students who participated in this study were 793 that is consisted in 268 third graders, 264 forth graders, and 261 fifth graders. Difficulty, reliability, discrimination index were analyzed and revised tests items to qualify standardized test. Cut off score was decided for using these test as diagnostic assessment. And that was confirmed two methods cooperatively norm referenced test as well as criterion referenced test. (The National Institute of the Korean Language) The purpose of this study is to develop the standardized instrument for diagnostic assessment of coursework in Korean Language Education. For this purpose, we develop the elements which can show the Korean Language Ability. We presented the elements of the Korean Language Ability that is related the 7th curriculum. The developed instrument was applied as a diagnostic assessment at two elementary schools in seoul. The students who participated in this study were 793 that is consisted in 268 third graders, 264 forth graders, and 261 fifth graders. Difficulty, reliability, discrimination index were analyzed and revised tests items to qualify standardized test. Cut off score was decided for using these test as diagnostic assessment. And that was confirmed two methods cooperatively norm referenced test as well as criterion referenced test. (The National Institute of the Korean Language)

교사의 학생평가전문성과 중등교사 양성과정

김신영(Kim Shinyoung) 한국교육평가학회 2007 교육평가연구 Vol.20 No.1

최근 이루어지고 있는 학교교육의 개혁은 교사들에게 수준 높은 학생평가전문성을 요구하고 있다. 교사의 평가전문성이 중요한 이유는 교사는 학생의 학업성취수준을 파악해 학습을 관리하고 모니터링해 줄 수 있어야 하기 때문이다. 한국교육과정평가원은 교사가 학생을 제대로 평가하기 위해 요구되는 평가지식과 기술을 중심으로 교사의 학생평가전문성기준을 개발하고 이에 기초한 교사의 학생평가전문성 계발 및 질 관리를 제안하고 있다. 이에 본 연구에서는 전문직의 조건과 교직의 비전문화 현상을 살펴보고 현재 중등교원 양성과정에서 학생평가전문성 계발을 위해 어떤 교육경험이 제공되고 있는지를 알아보았다. 또한 현장교사와 예비교사의 학생평가전문성 수준을 한국교육과정평가원이 개발한 교사의 학생평가전문성 진단도구로 진단해 보았다. 이는 중등교사양성과정에서 제공되어야 할 평가전문성 신장을 위한 교육경험의 구성과 교사의 학생평가전문성 신장방안의 모색에 의미있는 기초정보로 활용될 수 있을 것이다. The recent educational reform implemented nationwide requires that teachers possess a high level of student-assessment competency. Teachers’ competency of student assessment is important since it helps teachers to be prepared to monitor student learning and evaluate student achievement. However, most teachers frequently report that they are not fully prepared to undertake assessment-related activities. This may be mainly due to the fact that teachers receive little or no formal assessment training in the existing pre-service programs. This study has attempted to address the importance of teacher assessment competency by reviewing the current teacher certification programs and quantifying the level of teacher preparation in student assessment. The results indicate that most of the current educational evaluation courses do not provide adequate coverage for the assessment competency required of teachers. Even if they had, they were not compulsory. Based on the diagnostic tools developed by the Korea Institute of Curriculum and Evaluation(KICE), the level of student assessment competency of pre-service, as well as in-service teachers, were shown to be moderate. These findings indicate a significance for exploring and providing the educational experiences necessary to enhance the overall student-assessment competency levels of teachers.

For a More Useful EFL Cognitive Diagnostic Assessment

Yeon-Sook Yi(이연숙) 언어과학회 2016 언어과학연구 Vol.0 No.76

Cognitive diagnostic assessment can provide fine-grained diagnostic feedback and detailed scores reporting to the test-takers. Applying this assessment method to an EFL grammar test, this study aims to see if smaller grain size of grammar attributes can yield meaningful results of assessment. The study also looks into what kind of detailed information on student performance can be provided by utilizing the method. A set of response data to 23 grammar questions, collected from 434 college freshmen, was analyzed using a cognitive diagnostic model. The results present a positive answer to the research question of the feasibility of using smaller and more interpretable of grammar skills. The paper concludes with limitations of the current study and suggestions for future research.

영어 학습자 특성이 듣기 진단평가 결과에 미치는 영향 연구: 온라인대학을 중심으로

김주혜 이화여자대학교 교과교육연구소 2018 교과교육학연구 Vol.22 No.1

This study analyzed the effects of learner characteristics related to their English learning backgrounds on Listening Diagnostic Assessment results. To accomplish this aim, 283 Korean online university students were surveyed about the characteristics of English learners, and then a Listening Diagnostic Assessment of 10 subskills was conducted. Based on the results of the questionnaire and Listening Diagnostic Assessment, the effects of English learner characteristics on both the total score and the subscores of Listening Diagnostic Assessment subskills were investigated. The results of the study are as follows: First, the period of visits to English speaking countries, the time exposed to English audio/visual materials, English reading time, and conversation time with native English speakers were found to be significant factors that affected the total score of the Listening Diagnostic Assessment. Second, conversation time with native English speakers was found to be significant in the subscores of both the voice recognition and phrase/sentence identification subskills. Third, the period of visits to English speaking countries was found to be significantly related to the subscores of most subskills, and both the time exposed to English audio/visual materials and English reading time were found to be significant characteristics of the subscores of more than two subskills. Based on these results, practical suggestions related to listening courses were given to the instructors, the English contents developers, and the English curriculum developers of Online Universities. 본 연구는 영어학습 이력과 관련된 학습자 특성이 듣기 진단평가의 결과에 미치는 영향을 분석하였다. 연구목적달성을 위해 K온라인대학 영어강의 수강생 283명에게 영어 학습자 특성을 알아보는 설문조사를 한 후 10개의 소영역으로이루어진 듣기진단평가를 실시했다. 설문과 듣기진단평가 결과를 바탕으로 영어 학습자 특성이 듣기진단평가 총점수에 미치는 영향을 알아본 후, 각 소영역별 점수에 미치는 영향을 분석하였다. 연구결과는 다음과 같다. 첫째, 영어 학습자 특성에 따른 듣기 총점수의 차이를 비교해본 결과, 영어권 국가 방문기간, 영어 시청각자료 노출시간, 영어읽기 시간, 영어 원어민 대화시간 특성이 유의미한 차이를 보였다. 둘째, 초급단계 소영역인 소리식별과 어구 및 문장식별 소영역에서는 영어 원어민과의 대화시간이 공통적으로 의미 있는 차이를 보였다. 셋째, 대부분의 소영역에서는 영어국가 방문기간이 유의미한 특성이었으며, 영어 시청각자료 노출시간과 영어읽기 시간은 두 개 이상의 소영역에서 유의미한 것으로 나타났다. 이러한 결과를 바탕으로 온라인대학의 교수자, 영어 콘텐츠 개발자, 교육과정 기획자에게 듣기 수업과 관련된 실무적인 제안을 하였다.

Teachers’ Perceptions of Using the EDD Checklist in ESL Academic Writing Assessment: A Qualitative Study

김윤희 팬코리아영어교육학회 2011 영어교육연구 Vol.23 No.3

This study examined how teachers perceived the use of the Empirically-derived Descriptor-based Diagnostic (EDD) checklist and its diagnostic function in English as a second language (ESL) academic writing assessment. The EDD checklist was developed by Kim (2010) as a diagnostic tool for assessing ESL students’ academic writing performance. Ten ESL teachers assessed 480 Test of English as a Foreign LanguageTM Internet-based test (TOEFL® iBT) independent essays using the EDD checklist and were then interviewed about their experience using the checklist. The results showed that teachers generally considered the checklist to be an effective diagnostic assessment tool. They appreciated the checklist’s comprehensiveness and acknowledged that it covered multiple aspects of ESL academic writing. They also commented that the breakdown of writing skills helped them know what to look for when assessing essays. However, they pointed out that the checklist was time consuming and unable to fairly measure student writing abilities without taking essay length into consideration. While teachers cautioned that an appropriate amount of diagnostic feedback should be given to students, they also acknowledged that the use of the checklist could have a positive impact on classroom instruction. Pedagogical implications of the use of the EDD checklist in ESL writing classroom instruction and assessment are discussed.

Teachers’ Perceptions of Using the EDD Checklist in ESL Academic Writing Assessment : A Qualitative Study

Youn-Hee Kim 팬코리아영어교육학회 2011 영어교육연구 Vol.23 No.3

This study examined how teachers perceived the use of the Empirically-derived Descriptor-based Diagnostic (EDD) checklist and its diagnostic function in English as a second language (ESL) academic writing assessment. The EDD checklist was developed by Kim (2010) as a diagnostic tool for assessing ESL students’ academic writing performance. Ten ESL teachers assessed 480 Test of English as a Foreign LanguageTM Internet-based test (TOEFL?? iBT) independent essays using the EDD checklist and were then interviewed about their experience using the checklist. The results showed that teachers generally considered the checklist to be an effective diagnostic assessment tool. They appreciated the checklist’s comprehensiveness and acknowledged that it covered multiple aspects of ESL academic writing. They also commented that the breakdown of writing skills helped them know what to look for when assessing essays. However, they pointed out that the checklist was time consuming and unable to fairly measure student writing abilities without taking essay length into consideration. While teachers cautioned that an appropriate amount of diagnostic feedback should be given to students, they also acknowledged that the use of the checklist could have a positive impact on classroom instruction. Pedagogical implications of the use of the EDD checklist in ESL writing classroom instruction and assessment are discussed.

인지진단모형의 국어 교육적 접근 방안 탐색

최숙기 ( Choi Sook-ki ) 청람어문교육학회 2017 청람어문교육 Vol.0 No.62

인지진단모형을 적용한 인지진단평가(Cognitive Diagnostic Assessmen)는 기존의 규준 지향 검사들이 점수의 총합에 기반한 서열 및 순위 정보만 제공하는 문제를 극복하면서, 측정하고자 하는 능력의 세분화된 인지 요소의 숙달 상태에 따라 학습자의 학업적 강점과 약점을 진단한 뒤 이 에 대한 진단 정보를 학습자에게 제공하여 준다. 이를 통해 학습자는 학습 향상에 실질적 도움이 되는 교육적 피드백을 제공 받을 수 있을 뿐 아니라, 학습자 개별 학업 성취 프로파일 분석을 통해 맞춤형 학습 경로 설정하고 이에 대한 학업 계획을 수립할 수 있게 된다. 본 연구는 국어 평가 방안 개선을 위해 인지진단모형의 국어 교육적 적용 방안을 탐색하기 위해 선행 연구 탐색을 통해 인지진단평가의 시행 절차 및 수용 언어 기능 및 표현 언어 기능 측면에서 인지진단평가를 적용한 양상들을 체계적으로 고찰한 뒤 국어 교육적 적용을 위한 제언 사항을 도출하였다. Cognitive diagnostic assessment(CDA) can provide learners profile scores including information about their strengths and weaknesses in korean language assessment. CDA is different from traditional roles of norm-referenced assessment has been to rank-order examinees. CDA suggest strengths and weaknesses of individual learners in the target area of learning and instruction, such as korean language learning. And CDA provide authentic and effective diagnostic feedback about the current state of knowledge and skills of korean language learners so they can decide to appropriate lesson plan to remedy their weaknesses in various aspects of korean language ability. The aim of this study is to explore application Methods and challenges of cognitive diagnosis models in korean language assessment reviewing previous studies of L1 and L2 CDA.

논증적 글쓰기 인지진단평가를 위한 EDD(Empirically-derived and Descriptor-based Diagnostic) 체크리스트 개발 연구

박종임,최숙기,김태호 한국작문학회 2019 작문연구 Vol.0 No.41

This study focused on developing an 'assessment statement' for argumentative writing as a preliminary step to develop a Q-matrix that can be used in the Cognitive Diagnostic Assessment. In developing the 'assessment statement', the empirical validation procedure is used to overcome the limitation of the scoring criteria developed by relying only on the theoretical basis. 'Empirically-derived Descriptor-based Diagnostic' EDD) 'checklist. We also examined the possibility of developing a Q-matrix using an EDD checklist. In the follow-up research, we try to extract the cognitive factors for argumentative writing and validate the Q-matrix using the EDD checklist developed in this study. Through these studies, we hope to find ways to diagnose students' writing ability more specifically. 이 연구에서는 논증적 글쓰기 인지진단평가에서 사용할 수 있는 Q-행렬을 개발하기 위한 사전 단계로써 논증적 글쓰기에 대한 ‘평가 진술문’을 개발하는 것에 초점을 두었다. 또한 ‘평가 진술문’을 개발할 때에는 이론적 근거에만 의존하여 개발되어 온 채점기준의 한계를 극복하고자 실증적인 타당화 절차를 통해서 ‘실증적 평가 진술문 기반의 진단평가 (Empirically-derived Descriptor-based Diagnostic, 이하 EDD)’ 체크리스트 형태로 개발하고자 하였다. 또한 EDD 체크리스트를 활용하여 Q-행렬을 개발하기 위한 가능성을 살펴보았다. 후속 연구에서는 논증적 글쓰기에 대한 인지요소를 추출하고 본 연구에서 개발한 EDD 체크리스트를 활용하여 Q-행렬을 타당화하는 작업을 수행하고자 한다. 이러한 연구를 통해서 학생들의 쓰기 능력을 보다 구체적으로 진단할 수 있는 방안이 마련되기를 기대해 본다.

핵심역량 진단도구 개발 연구 : Y 대학 사례 중심으로

이윤주(Lee, Yoonjoo),이용상(Lee, Yongsang),문명현(Moon, Myunghyun) 학습자중심교과교육학회 2020 학습자중심교과교육연구 Vol.20 No.8

본 연구는 Y대학의 인재상에 부합하는 핵심역량의 개념을 정립하고 이를 측정하기 위한 진단도구를 개발하는데 그 목적이 있다. 먼저 선행연구 분석과 면담 등을 통해 Y대학의 5대 핵심역량을 구체화 하고, 각 역량별로 3개의 하위요인을 구성하였다. 이를 기초로 선행연구 분석, 인터뷰 등을 통해 159문항을 개발하여 예비검사를 실시하였다. 예비검사 자료를 바탕으로 요인 분석을 실시하여 요인 부하량의 절대값이 0.4이하인 문항과 문항반응모형 분석 결과 모형 적합도가 0.7이하 또는 1.3 이상인 문항을 제거한 후 요인별로 내적일관성 신뢰도가 0.7이상이 되는지를 검증하였다. 이러한 절차를 거쳐 총 133문항을 선제하여 본검사를 시행하였으며, 본검사 자료를 바탕으로 검사도구의 신뢰도와 타당도를 검증하였다. 본 연구에서는 이를 위해 우선 문항 평균과 표준편차를 확인하여 문항 평균의 분포가 양호함으로 검증하였고, 요인분석을 통해 구인타당도가 양호한 것을 확인하였다. 다음으로 요인별, 역량별, 진단도구 내적일관성 신뢰도를 분석한 결과 대부분의 요인, 역량에 있어서 신뢰도가 0.8 이상으로 높은 것으로 나타났다. 마지막으로 문항반응모형 분석을 통해 문항 적합도를 검증하였으며, 핵심역량 진단도구가 학생들의 전체 역량 수준을 적절하게 측정하고 있는지 확인함과 동시에 5점 리커르트 척도가 학생들의 역량 수준을 잘 변별하고 있음을 확인하였다. 본 연구를 통하여 개발된 Y대학의 핵심역량 진단도구가 신뢰도와 타당도를 갖춘 측정도구임을 확인할 수 있었으며, 연구결과를 토대로 Y대학 핵심역량 진단도구가 갖는 의의와 활용방안을 논의하였다. The purpose of this study is to develop diagnostic assessment tools to measure students’ core competencies in Y University. First, the five core competencies of Y University were specified through a prior study analysis and interviews. Based on this, 159 items were developed, and then examined with preliminary surveys. The preliminary survey was conducted by 723 students of Y University. With preliminary data, the factors analysis, reliability analysis, and item response model analysis were conducted to examine items. Based on analysis results, items with loading lower than 0.4 had been deleted, and then item fit statistics had been examined for further deletion. Eventually total 133 items were selected for the main survey. In the main survey empirical data was obtained from 455 students and analyzed for the validation of the diagnostic assessment tools. To this end, the mean and the standard deviation was checked first to verify that the distribution of the mean of the item was good, and confirmatory factor analysis was implemented to test construct validity. Next internal consistent reliability was examined, and then item response model analysis was finally conducted to see whether our diagnostic assessment tools properly measure the whole range of students’ core competencies. This anlaysis results also showed the 5-point Likert scale is properly working in our assessment. The empirical analysis results clearly show our diagnostic assessment tools are valid and reliable enough to measure Y University core competencies. Finally the significance and application of core competency diagnostic assessment tools for Y University was discussed.