수용성 판단 실험의 설계 및 통계 분석 처리의 몇 문제- 서열형·연속형 데이터를 중심으로 -

조용준 한말연구학회 2023 한말연구 Vol.64 No.42

. Acceptability judgment experiments, often conducted as surveys, offer the advantage of obtaining a large amount of data with minimal time, effort, or cost. However, conducting acceptability judgment experiments involves several considerations. First is the issue related to the sensitivity of result analysis, second is related to the identification of careless responses, and third pertains to the statistical analysis of result data. This study empirically investigates these issues, focusing on ordinal data and continuous data. Sensitivity, represented by statistical power, was assessed across various effect sizes with sample sizes ranging from 5 to 100, each simulated with 1,000 resamplings. Similar to Sprouse & Almeida 2017, for the Likert scale(LS), it was found that a minimum of 9 participants is required for an extra-large effect size, 19 for a large effect size, and 51 for a medium effect size. For the magnitude estimation(ME), in the case of an extra-large effect size, at least 11 participants are needed, 25 for a large effect size, and 75 for a medium effect size. In both methods, for a small effect size, statistical power did not reach 80%. Additionally, relying on crowd-sourced large-scale acceptability judgment experiments, 30 standard sentences (gold standard) for identifying careless responses were provided, consisting of 30 grammatical and 30 ungrammatical sentences. These could be useful for developing practice items and items for identifying careless responders. Moreover, 20 sentences are presented for designing practice items, for participants to grasp a possible range of of acceptability judgments. This is expected to be very helpful in designing acceptability judgment experiments. Finally, we examined various possible statistical analyses for ordinal and continuous data. For ordinal data, there are methods such as standardization for statistical analysis or multivariate cumulative logit analysis using raw values. For continuous data, there are methods such as standardization or natural logarithm transformation for statistical analysis. As an example, for each of the four syntactic phenomena, 200 samples were randomly selected, and statistical analyses were compared. Overall, we suggest that standardizing ordinal data for statistical analysis is preferable.

What do you mean by contrast in syntax?

( Sanghoun Song ),( Eunjeong Oh ) 경희대학교 언어정보연구소 2017 언어연구 Vol.34 No.3

The present work proposes a non-binary evaluation of contrast in syntax and elaborates the benefits of considering gradience and degree in syntactic study. Contrast is one of the most important notions in contemporary linguistics, but the field lacks consensus about its definition and its role. Syntactic contrast has long been used in the field as a binary means of distinguishing grammatical from ungrammatical sentences via introspective judgments. Given that not all (un)grammatical sentences sound equally good or bad, contrasts should also be posited as gradient, thereby being measurable on a continuum of acceptability. The present study argues that a gradient view of syntactic contrasts is often more informative, revealing a greater variety of syntactic underpinnings in human language. To substantiate the merits of the gradient view of contrast over the dichotomous view, the present study presents results from a series of experiments conducted on Korean. The test items consist of 287 sentence pairs randomly extracted from Studies of Generative Grammar 1991-2014. The experimental tasks include a two-alternative forced choice task, a binary yes/no task, and a 5-point Likert scale task. The analysis is four-pronged covering direction, position, distance, and intensity of contrast. First, the direction of syntactic contrasts is examined with respect to whether linguists’ judgments and naive speakers’ judgments converge with each other. Second, the position of contrasts on acceptability continuum examines absolute goodness of sentences on the assumption that the ‘good’ and ‘bad’ sentence must absolutely sound good and bad to the vast majority of speakers. Third, the distance of contrasts pertains to strength of grammatical constraints. This indicates the magnitude of difference in acceptability between two pairwise sentences. Finally, the intensity of contrasts relates to judgment variation across speakers. This examines whether most naive speakers agree with the acceptability of a particular sentence and if not, it estimates how much their judgments are scattered in terms of the gradience of acceptability. (Incheon National University · Sangmyung University)

What do you mean by contrast in syntax?

송상헌,오은정 경희대학교 언어정보연구소 2017 언어연구 Vol.34 No.3

The present work proposes a non-binary evaluation of contrast in syntax and elaborates the benefits of considering gradience and degree in syntactic study. Contrast is one of the most important notions in contemporary linguistics, but the field lacks consensus about its definition and its role. Syntactic contrast has long been used in the field as a binary means of distinguishing grammatical from ungrammatical sentences via introspective judgments. Given that not all (un)grammatical sentences sound equally good or bad, contrasts should also be posited as gradient, thereby being measurable on a continuum of acceptability. The present study argues that a gradient view of syntactic contrasts is often more informative, revealing a greater variety of syntactic underpinnings in human language. To substantiate the merits of the gradient view of contrast over the dichotomous view, the present study presents results from a series of experiments conducted on Korean. The test items consist of 287 sentence pairs randomly extracted from Studies of Generative Grammar 1991-2014. The experimental tasks include a two-alternative forced choice task, a binary yes/no task, and a 5-point Likert scale task. The analysis is four-pronged covering direction, position, distance, and intensity of contrast. First, the direction of syntactic contrasts is examined with respect to whether linguists’ judgments and naïve speakers’ judgments converge with each other. Second, the position of contrasts on acceptability continuum examines absolute goodness of sentences on the assumption that the ‘good’ and ‘bad’ sentence must absolutely sound good and bad to the vast majority of speakers. Third, the distance of contrasts pertains to strength of grammatical constraints. This indicates the magnitude of difference in acceptability between two pairwise sentences. Finally, the intensity of contrasts relates to judgment variation across speakers. This examines whether most naïve speakers agree with the acceptability of a particular sentence and if not, it estimates how much their judgments are scattered in terms of the gradience of acceptability.

In Defense of Comparisons between Formal and Informal Acceptability Judgments

Sang houn Song,Eun jeong Oh 한국생성문법학회 2017 생성문법연구 Vol.27 No.4

This squib addresses the theoretical and empirical premise of comparisons between informal and formal acceptability judgments in defense of necessities and implications of our own work on experimental syntax. Since the early days of experimental syntax, some linguists have expressed concerns about comparing the results of informal and formal methods. Contrary to this stance, we have found ample evidence that comparing these two types of judgments is not only justified but also a fruitful method for delving into syntactic constraints.

In Defense of Comparisons between Formal and Informal Acceptability Judgments

송상헌,오은정 한국생성문법학회 2017 생성문법연구 Vol.27 No.4

This squib addresses the theoretical and empirical premise of comparisons between informal and formal acceptability judgments in defense of necessities and implications of our own work on experimental syntax. Since the early days of experimental syntax, some linguists have expressed concerns about comparing the results of informal and formal methods. Contrary to this stance, we have found ample evidence that comparing these two types of judgments is not only justified but also a fruitful method for delving into syntactic constraints.

補助用言 構文의 統合 關係: 實驗 言語學的 接近-‘-고 말-’ 구문을 중심으로 -

명정희(明廷姬) ( Myeong Jeong-hee ),구호정(具湖定) ( Koo Ho-jeong ) 한국어문교육연구회 2024 어문연구(語文硏究) Vol.52 No.1

한국어 보조용언 구문은 두 개 이상이 통합되어 나타날 수 있는데 통합 관계에 대한 직관이 논자에 따라 상이한 경우가 많아 통합 관계를 논의하기 쉽지 않다. 이에 이 연구에서는 ‘-고 말-’ 구문을 중심으로 보조용언 구문 통합 관계의 실재를 실험을 통해 확인한 후, 한국어 보조용언 구문의 통합 제약에 대한 이론적 설명을 제시하였다. 한국어 모어 화자들을 대상으로 실험한 결과, ‘-고 말-’ 구문은 대체로 다른 보조용언 구문들과 통합할 때 후행하는 것이 선호되는데, ‘-을 수 있-’ 구문과 통합할 때에는 ‘-고 말-’ 구문이 선행하는 것이 선호되었다. 이는 ‘-고 말-’ 구문이 표현적 의미로 해석될 때 넓은 영향권을 가져 생기는 ‘후행성 제약’으로 설명할 수 있다. 한편 ‘-고 말-’ 구문은 양태 구문 ‘-고 싶-’, ‘-어야 하-’와 진행상의 ‘-고 있-’ 구문과 통합할 때에는 순서에 따라 수용성이 크게 달라지지 않으며 전반적으로 수용성이 낮게 나타났다. 이는 ‘-고 말-’ 구문의 ‘종결성 제약’으로 설명할 수 있다. In Korean, auxiliary verb constructions can occur with two or more integrated constructions, leading to varying intuitions in scholars’ judgments of the acceptability of examples. This study examines the syntagmatic order of auxiliary verb constructions, focusing on the ‘-go mal-’ construction, and provides theoretical explanations for the integration constraints of Korean auxiliary verb constructions. The results of experiments conducted with native Korean speakers indicate a preference for the ‘-go mal-’ construction to be post-positioned when integrating it with other auxiliary constructions, whereas pre-posing is preferred when it is integrated with the ‘-eul su iss-’ construction. This preference can be explained by the “posteriority constraint” that arises when interpreting the ‘-go mal-’ construction in terms of expressive meaning. Conversely, the ‘-go mal-’ construction shows less sensitivity to order when integrated with modal constructions such as ‘-go sip-’ and ‘-eoya ha-’, and with the progressive aspect ‘-go iss-’, exhibiting low acceptability overall. This phenomenon arises from the ‘-go mal-’ construction presupposing a telic event, which can be explained by the “termination constraint.”