국어교사의 총체적, 분석적 쓰기 채점과 평가자 협의 과정 연구 = Research on the Process of Holistic and Analytic Scoring and Rater Discussion of Korean Teachers

다국어 초록 (Multilingual Abstract)

The aim of this study is about holistic and analytic scoring process of raters, the use of the writing assessment criteria and the rater`s discussion process. The raters express cognitive activities of writing assessment with the think aloud method and the writing comment. And this study explores the features of the process of the rater`s discussion on the two groups with the discourse analysis method. This study specifically analyzes the process of using the writing assessment criteria and the rater`s discussion by simple regression analysis. The result of this study shows the differences of the scoring groups on the scoring and the use of writing assessment criteria. The result of analysis about the process of the rater`s discussion shows the importances of the balance between the agreement and the disagreement. Therefore considering of cognitive scoring activities of raters and using the writing assessment criteria makes an important contribution to raising the quality of writing assessment. And consideration of the rater`s discussion makes an important contribution to raising the quality of (pre)teacher writing assessment education.

참고문헌 (Reference)

1 김정숙, "한국어 쓰기 능력 평가 방안- 종합적 채점과 분석적 채점 결과를 중심으로" 이중언어학회 (43) : 81-99, 2010

2 이승재, "평가자 협의가 작문평가의 신뢰도에 미치는 영향" 한국교원대학교 대학원 2011

3 박영민, "평가 예시문을 활용한 쓰기 평가 개선 방안" 청람어문교육학회 (39) : 111-133, 2009

4 성태제, "타당도와 신뢰도" 학지사 1995

5 손다혜, "총체적 쓰기 평가와 분석적 쓰기 평가의 결과 비교 연구 : 고등학생 논설문, 설명문, 수필을 중심으로" 연세대학교 교육대학원 2011

6 박영민, "중학생 논설문 평가의 모평균 추정과 평가 예시문 선정" 한국어교육학회 (131) : 437-461, 2010

7 김정덕, "중학교 쓰기 수업과 평가를 위한 루브릭 사례 연구" 한양대학교 대학원 2009

8 조재윤, "일반화가능도 이론을 이용한 쓰기 평가의 오차원 분석 및 신뢰도 추정 연구" 한국어교육학회 (128) : 325-358, 2009

9 박영민, "예비국어교사의 중학생 논설문 평가에서 발견되는 엄격성 및 일관성의 특성" 국어교육학회 (43) : 253-283, 2012

10 배향란, "쓰기의 총체적 평가 방법 연구" 韓國敎員大學校 大學院 1995

11 서수현, "쓰기 평가 협의 과정에 나타난 쓰기 평가자의 인식 연구" 국어교육학회 (44) : 335-367, 2012

12 박영목, "쓰기 평가 연구의 주요 과제" 한국작문학회 (6) : 9-38, 2008

13 이순영, "상징적 표상법의 이론과 실제" 국어교육학회 (32) : 341-367, 2008

14 노명완, "문식성 교육 연구" 한국문화사 13-41, 2008

15 오세영, "디지털 시대 국어교사의 인지활동과 쓰기 평가의 상관 연구" 한국국어교육학회 (92) : 255-290, 2012

16 정희모, "대학생 글에 대한 총체적 평가와 분석적 평가의 결과 비교 연구" 청람어문교육학회 (39) : 251-273, 2009

17 김미영, "논술문 평가의 신뢰도 향상에 관한 연구 : 평가자 협의회를 중심으로" 고려대학교 교육대학원 2004

18 정희모, "글쓰기 평가에서 객관-주관주의 대립과 그 함의" 우리어문학회 (37) : 217-249, 2010

19 Colombini, C. B., "“Storming and norming”: Exploring the value of group development models in addressing conflict in communal writing assessment" 17 (17): 191-207, 2012

20 Vacc, N. N., "Writing evaluation: Examining four teachers' holistic and analytic scoring" 90 (90): 87-95, 1989

21 Greenberg Karen, L., "Validity and reliability issues in the direct assessment of writing" 16 : 7-22, 1992

22 Huot, B. A., "Toward a new theory of writing assessment" 47 (47): 549-566, 1996

23 Wolfe, E. W., "The relationship between essay reading style and scoring proficiency in a psychometric scoring system" 4 (4): 83-106, 1997

24 Huot, B. A., "The influence of holistic scoring procedures on reading and rating student essays, In Validating holistic scoring for writing assessment" Hampton Press 206-236, 1993

25 Mennecke, B. E., "The implications of group development and history for group support system theory and practice" 23 (23): 524-572, 1992

26 Huot, B. A., "The Literature of Direct Writing Assessment: Major Concerns and Prevailing Trends" 60 (60): 237-263, 1990

27 Johnston, B., "Summative assessment of portfolios: An examination of different approaches to agreement over outcomes" 29 (29): 395-412, 2004

28 Tuckman, B. W., "Stages of small-group development revisited" 2 (2): 419-427, 1977

29 Moss, P. A., "Shifting conceptions of validity in educational measurement: Implications for performance assessment" 62 (62): 229-258, 1992

30 Huot, B. A., "Reliability, validity, and holistic scoring: What we know and what we need to know" 41 : 201-213, 1990

31 Burnham, C., "Reflection, assessment, and articulation: A rhetoric of writing program administration, In The writing program administrator’s resource: A guide to reflective institutional practice" Lawrence Erlbaum Associates, Inc 303-314, 2002

32 Knoch. U., "Re-training writing raters online: How does it compare with face-to-face training?" 12 (12): 26-43, 2007

33 박종임, "Rasch 모형을 활용한 국어교사의 채점 일관성 변화 양상 및 원인 분석 -중학생 서사문 채점을 중심으로-" 우리어문학회 (39) : 301-335, 2011

34 박영민, "Rasch 모형을 활용한 국어교사의 쓰기 평가 특성 분석－중학생 설명문 쓰기 평가를 중심으로－" 국어교육학회 (37) : 367-391, 2010

35 Broad, B., "Pulling your hair out: Crises of standardization in communal writing assessment" 35 (35): 213-260, 2000

36 Larson, R. L., "Portfolios in the assessment of writing: A political perspective, In Assessment of writing: Politics, policies, practices" MLA 271-283, 1997

37 Messick, S., "Meaning and values in test validation: The science and ethics of assessment" 8 (8): 5-11, 1989

38 Yancey. K. B., "Looking back as we look forward: Historicizing writing assessment" 50 (50): 483-503, 1999

39 Weigle, S. C., "Investing Rater/Prompt Interactions in Writing Assessment: Quantitative and Qualitative Approaches" 6 (6): 145-178, 1999

40 Knoch, U., "Investigating the effectiveness of individualized feedback to rating behaviour –A longitudinal study" 28 : 179-200, 2011

41 Vaughan, C., "Holistic assessment: What goes on in the rater’ mind?, In Assessing second language writing in academic contexts" Ablex 111-125, 1991

42 Freedman, S. W., "Holistic assessment of writing: Experimental design and cognitive theory, In Research on writing: Principles and methods" Longman 75-98, 1983

43 Thompson, R. W., "Holistic assessment of a high school writing skills curriculum" s. n 1991

44 Smith, G., "Group development: A review of the literature and a commentary on future research directions" 3 : 14-46, 2001

45 East, M., "Evaluating the reliability of a detailed analytic scoring rubric for foreign language writing" 14 : 88-115, 2009

46 de Lichtenberg, J., "Evaluating group interventions: A framework for diagnosing, implementing, and evaluating group interventions" 9 : 37-48, 2008

47 Cooper, C. R., "Evaluating Writing : describing, Measuring, Judging" National Council of Teachers of English 1977

48 Tuckman, B. W., "Developmental sequence in small groups" 63 (63): 384-399, 1965

49 Ruth, L., "Designing writing tasks for the assessment of writing" Ablex Publishing Corporation 1988

50 Spendel, V., "Creating writers: Linking assessment and writing instruction" Longman 1997

51 Murphy, S., "Construct and Consequence：Validity in Writing Assessment, In Hand book of Research of writing ： History, Society, School, Individual" IEA 2008

52 Trimbur, J., "Consensus and difference in collaborative learning" 51 (51): 602-616, 1989

53 Mislevy. R. J., "Comments on lissitz and samuelsen: Validity by design" 36 (36): 463-469, 2007

54 Moss. P. A., "Comments on lissitz and samuelsen: Reconstructing validity" 36 (36): 470-476, 2007

55 Lynne, P., "Coming to terms: A theory of writing assessment" Utah State UP 2004

56 Hayes, J. R., "Cognitive processes in revision, In Advances in applied psycholinguistics: Vol. 2. Reading, writing, and language processing" Cambridge University Press 176-240, 1987

57 Camp, R., "Changing the model for the direct assessment of writing, In Validating holistic scoring for writing assessment: Theoretical and empirical foundations" Hampton 45-78, 1993

58 Tierney, R. D., "Changing practices: Influences on classroom assessment" 32 (32): 239-264, 2006

59 Lumley, T., "Assessment criteria in a large-scale writing test: What do they really mean to the raters?" 19 (19): 246-276, 2002

60 Lumley, T., "Assessing second language writing: The rater’s perspective" Lang 2005

61 Petruzzi, A., "Articulating a hermeneutic theory of writing assessment" 13 : 219-242, 2008

62 Wolfe, E. W., "A study of expertise in essay scoring" University of California 1995

63 Huot, B. A., "(Re)articulating writing assessment for teaching and learning" Utah State University Press 2002