대학 강의평가도구의 타당화 및 관련 변인의 다층분석

염시창(Yum Sichang) 한국교육평가학회 2008 교육평가연구 Vol.21 No.2

  본 연구는 대학에서 학기말에 활용할 수 있는 다차원적 강의평가도구의 일차원 요인과 이차원 요인의 확인을 통해 평가도구를 타당화하고, 강의총평에 대한 강의평가 요인의 영향력을 분석하며, 강의총평 관련 변인의 효과를 다층분석을 이용하여 검증하기 위해 수행되었다. 본 연구에서는 25문항으로 구성된 강의평가도구를 대학생 5,575명이 수강한 413과목에 시행하여 총 32,670명의 자료를 분석하였다. 본 연구에서 나타난 결과는 다음과 같다. 첫째, 탐색적ㆍ확인적 요인분석 결과, ‘강의의 전반적 질’ 영역에 강의구성 및 강의진행 요인, 그리고 ‘학습자 규제’ 영역에 평가와 강의부담 요인이 포함된 것으로 나타났다. 그러나 이차원 이요인 모형과 이차원 일요인 모형을 비교한 결과, 두 가지 모형의 적합도에 차이가 없는 것으로 나타났다. 둘째, 강의총평에 대한 영향력 분석에서는 강의구성, 강의진행, 평가 요인의 영향력이 유의하였다. 셋째, 강의총평 관련 변인의 다층분석 결과, 학생수준의 기대학점과 강좌수준의 강의구성 및 강의진행 변인의 영향력이 유의한 반면, 교과구분 및 강의부담 변인은 유의하지 못하였다. 또한 기대학점과 강의구성 및 강의진행 변인에 의해 강의총평 분산이 유의하게 설명되었다. 따라서 기대학점의 영향력을 통제할 때, 강의총평의 합성점수가 총괄적 강의평가 점수로 활용될 수도 있다는 점을 보여 주었다.   This study examined the validity of a students" ratings of teaching scale through a series of factor analyses as well as structural and multilevel modeling. Students" ratings of teaching scale contained 25 evaluative items, which were used for collecting data from 5,575 students with 413 courses. The exploratory and confirmatory factor analyses provided a clear interpretation of first-order teaching quality on four factors: course organization/design, pedagogic performance, grading and course demands. However, the results did not provide a clear interpretation of two second-order dimensions of teaching quality: general quality of teaching and regulating students" learning. Scores on the course organization/design and the pedagogic performance factors had the strong positive relationship with the factor of overall rating on the scale. A series of multilevel modeling were used to answer the impact of the student- and the course-level variables on overall rating. The outcomes from these models revealed that expected grade as well as course organization/design and pedagogic performance had significant impact on overall rating. In these models, the between-course variances of overall rating were smaller and became statistically insignificant. This finding showed that course organization/design and pedagogic performance predicted overall rating when potential impact of expected grade was ruled out, indicating that a composite score of overall rating could be used for summative purpose.

성장혼합모형을 활용한 초등학생의 수학 학업성취도 변화 유형화 및 영향요인 분석

염시창(Sichang Yum),강대중(Daejung Kang) 전남대학교 교육문제연구소 2023 교육연구 Vol.45 No.3

웹기반 하이퍼텍스트 읽기부진 구인 탐색 및 정신모형구축에 대한 영향

김회수 ( Hoisoo Kim ),염시창 ( Sichang Yum ),신현숙 ( Hyeonsook ),( Thurlow M. ) 한국교육공학회 2004 교육공학연구 Vol.20 No.4

The purpose of this study was to identify the construct of the hypertext reading underachievement. One-hundred fourteen elementary school students and 95 college students participated in this study. The elementary school students were randomly assigned to two conditions: the hypertext condition(n=59), the linear text condition(n=55). The college students were all assigned to the hypertext condition. Reading underachievers(UA) were defined as those who earned scores above the median on the test of prior knowledge but earned scores below the median on the test of post comprehension. Those who earned scores above the median on the test of prior knowledge and the test of post comprehension were classified into maintenance readers(MR). In contrast, those who earned scores below the median on the test of prior knowledge but earned scores above the median on the test of post comprehension were labelled the improved readers(IR). The multivariate analyses of variance(MANOVA) were conducted with the reading groups as the independent variable and various cognitive measures(e.g., working memory, critical thinking as the dependent variables. Also, the multivariate analyses of covariance(MANCOVA) were conducted with the reading groups as the IV, various cognitive measures(e.g., recall, concept maps) as the DVs, and the scores of prior knowledge as the covariate. The results of this study were as follows: First, the results of the MANOVA showed that among college students, the three groups of hypertext reading (UA, MR, IR) did not differ in the scores of working memory and critical thinking. Second, the results of the MANCOVA showed that among college students, the group effect was significant in the scores of recall and concept maps. In recall, the IR was higher that the MR and the UA. On the test of concept maps, the IR scored the highest and the MR scored higher than the UA, which scored the lowest. Third, among elementary school students, the three groups of hypertext reading (UA, MR, IR) did not differ in the level of working memory; however, they significantly differed in the level of critical thinking. Post-hoc tests revealed that MR scored higher than the UA. Fourth, the results of MANCOVA showed that among elementary school students, three hypertext reading groups significantly differed both on the text-based comprehension and the situation model comprehension. That is, the IR scored the highest on both types of comprehension, the MR scored the next, and the UA scored the lowest. Fifth, among elementary school students, the three groups of linear text reading (IR, MR, UA) did not differ in two components of working memory such as central executive(CE) and phonological loop(PL), but differed in the other component such as visuospatial sketchpad(VSSP). In VSSP, the IR and the UA scored higher than the MR. Sixth, among elementary school students, the three groups of linear text reading (IR, MR, UA) did not differ in critical thinking. The result of this study imply that the construct of reading underachievement does exist even though reading underachievers did not differ in working memory and critical thinking from maintenance readers and improved readers.

하이퍼텍스트 보조 학습에서 사전지식, 작동기억 및 하이퍼텍스트 탐색도구 유형이 학업성취도에 미치는 효과

김회수 ( Hoisoo Kim ),염시창 ( Sichang Yum ),( Diane L. Schallert ) 한국교육공학회 2002 교육공학연구 Vol.18 No.1

The purpose of this study was to examine the effects of students` prior knowledge and working memory, and the hypertext navigation tools on performance with hypertext. According to Baddeley (1986), it was considered that the concept of working memory was divided into three components such as central executive, phonological loop, and visuospatial sketchpad. The hypertext navigation tools included four types: the table of content, the list, the structural map, and the structural map with its introduction. The subject was 118 college students. They were randomly assigned to the four hypertext navigation conditions. The research findings were as follows: First, students` prior knowledge had the significant effect on performance with hypertext even if the main effect of the hypertext navigation tools was not statistically significant. The interaction effect of the prior knowledge and the types of hypertext navigation on performance, however, was significant. Second, three components of working memory had effects on performance with hypertext. Both phonological loop and visuospatial sketchpad affected the subjects` performance. The central executive measured by the star counting task had significant effect on performance, but the central executive measured by the random number generation task did not have the main effect on the achievement. Third, the students of high phonological loop capacity showed higher posttest scores in the two conditions of the list and the table of content, even though the low capacity students had higher scores in the conditions of the structural map and the structural map with its introduction. Similarly, the students of high visuospatial sketchpad capacity had higher posttest scores than the lower students in both the list and the table of content conditions. However, there was no significant difference between the two groups in the conditions of the structural map and the structural map with introduction.

초등학교 교사들의 형성평가에 대한 인식 및 실태 분석

강대중(Kang, DaeJung),염시창(Yum, Sichang) 학습자중심교과교육학회 2014 학습자중심교과교육연구 Vol.14 No.2

이 연구는 초등학교 교사들이 형성평가를 어떻게 인식하고 실행하며, 그 결과를 어떻게 활용하고 있는가를 분석하기 위해 수행되었다. 이를 위해 G광역시에 근무하고 있는 초등학교 교사 333명의 데이터를 활용하여 교사의 성별, 평가연수 이수 여부 및 교육경력에 따른 인식의 차이를 분석하였다. 분석 결과는 다음과 같다. 첫째, 평가 연수를 이수한 교사 집단이 그렇지 않은 집단에 비해 평가 자신감과 평가 전문성이 높은 것으로 나타났는데, 이는 평가 연수가 초등학교 교사들의 평가 전문성과 자신감에 실질적인 영향을 미치고 있음을 시사한다. 둘째,교육경력이 많을수록 형성평가를 수업 후반부에 학습 내용을 확인하는 평가라고 생각한 비율이 높았고 형성평가 실시 시기에 관한 인식에서도 절반 이상의 교사들이 형성평가를 수업 후반부에 실시한다고 응답하였다. 이러한 결과를 보면 많은 교사들이 아직도 형성평가를 학생들의 학습을 위한 평가라기보다는 등급을 결정하는 규준 지향적인 평가라고 인식하고 있음을 짐작할 수 있다. 셋째, 형성평가의 구체적인 방법 중에서 쪽지시험을 포함한 시험지를 가장 많이 사용하였으며, 그 다음으로 발문, 교과서 문제 해결, 학생 거수 순이었다. 넷째, 많은 교사들이 발문 후 학생들의 답변을 기다리는 시간이 5초 이내였다. 다섯째, 교사들은 학생들에게 정답 이외의 추가적인 정보까지 피드백하고 있었으며, 피드백의 방법으로는 교사와 학생간의 약속된 기호 표시가 가장 많았다. 마지막으로 이 연구에서는 형성평가의 발전적인 방향으로 학생의 반응에 근거한 피드백을 강조하는 실질적인 평가 연수 기회 제공 등의 정책적 방안을 제시하였다. The purpose of this study was to find out how elementary school teachers perceived and implemented formative assessments. For this purpose, this study analyzed the data of 333 elementary school teachers currently working in G metropolitan city. The result of the analysis was as follows. First, teachers who were trained in evaluation displayed more evaluation pride and higher evaluation literacy than those who had no experience in evaluation training. The result showed that evaluation training had substantive effects on the teachers’ evaluation pride and evaluation literacy. Second, teachers’ career in education correlated with their perception of formative assessment as a tool to check the contents of the curriculum in the end of a lesson. In addition, the questionnaires on the period of formative assessment revealed that more than half of the teachers recognized that the formative assessment should take place at the end of the class. This result indicated that many teachers still perceived the formative assessment as a norm-referenced assessment for grading, rather than an assessment for learning. Third, the study showed that in terms of the specific methods for formative assessments, examinations, including quizzes, were the most frequently used method. Fourth, a majority of teachers waited for five seconds to receive answers after questioning them. Fifth, teachers gave feedback on additional information aside the correct answer to the posed questions. Signals previously promised between the teachers and students were the most frequently used method for feedback. This study proposed that evaluation training emphasizing feedback would help teachers expand assessment for learning.

Rogers의 혁신확산이론에 기초한 교사의 e-러닝 정책 수용·확산 검사 개발

김회수 ( Kim Hoisoo ),( Chi Fushu ),( Zheng Xiaoli ),염시창 ( Yum Sichang ) 한국교육정보미디어학회 2018 교육정보미디어연구 Vol.24 No.1

This paper aims to develop and verify (1) the inventory which measures the level of teachers’ adoption and diffusion of e-learning policies (TADE) and (2) the inventory which measures teachers’ perceived characteristics of e-learning (TCE) such as relative advantage, compatibility, complexity, trialability, and observability on the basis of Rogers’ diffusion of innovation theory. Participants were 1,097 school teachers, consisting of 532 Chinese teachers and 558 Korean teachers. The inventories are developed and verified by the process of examination of related previous research, experts’ review, and exploratory factor analysis. TADE consists of 21 items under five factors: Teachers’ level of e-learning use, teachers’ level of recommendation of student’s e-learning use, teachers’ willingness of digital textbook use, teachers’ level of adoption of e-learning policy, teachers’ willingness of e-learning diffusion. Alfa coefficient of TADE is .98. Alpha coefficients of 5 sub-factors of TADE range from .73 to .98. TCE consists of 14 items under five factors: Relative advantages, compatibility, simplicity, trialability, observability. Alfa coefficient of TCE is .96. Alpha coefficients of the 5 factors of TCE range from .72 to .96. The inventories developed in this study is found to have both content validity and reliability for assessing teachers’ adoption and diffusion of e-learning in Korea and China.