지원사업
학술연구/단체지원/교육 등 연구자 활동을 지속하도록 DBpia가 지원하고 있어요.
커뮤니티
연구자들이 자신의 연구와 전문성을 널리 알리고, 새로운 협력의 기회를 만들 수 있는 네트워킹 공간이에요.
논문 기본 정보
- 자료유형
- 학술저널
- 저자정보
- 저널정보
- 한국언어과학회 언어과학 언어과학 제28권 제4호
- 발행연도
- 2021.11
- 수록면
- 301 - 317 (17page)
- DOI
- http://dx.doi.org/10.14384/kals.2021.28.4.301
이용수
초록· 키워드
오류제보하기
Since the introduction of BERT, recent works have shown success in detecting when a word is anomalous given sentence context. Since likelihood score is not an appropriate tool in identifying the exact property of linguistic anomaly, Li et al. (2021) recently adopt Gaussian models for density estimation at intermediate layers of pretrained language models. They find that different English pretrained language models employ separate mechanisms to recognize different types of linguistic anomaly. In keeping with Li et al.‘s methodology, we probe whether Korean counterparts such as KoBERT and KR-BERT are sensitive to different levels of linguistic anomaly, just as English-based language models are. To investigate the issue concerned, we construct an experiment with a suite of test data involving morphosyntactic, semantic, and commonsense anomaly in Korean and apply the two Korean-based models to test relevant sentences. We find that KoBERT and KR-BERT show relatively higher surprisal gaps throughout layers when the anomaly is morphosyntactic than when the anomaly is semantic. By contrast, commonsense anomaly does not exhibit any surprisal gap in any layer. We thus report that, like their English counterparts, KoBERT and KR-BERT use different mechanisms to track the different types of linguistic anomaly.
목차
등록된 정보가 없습니다.
참고문헌 (20)
참고문헌 신청
Choi, S. J. / 2021 / KoBERT-wa KR-BERT-uy unnikchungpyel thongsa mich uymi cheli sengnung phyengka / Annual Conference on Human and Language Technology. Human and Language Technology :340~345
Chomsky, N. / 1957 / Syntactic Structures / Mouton
Chow, W. Y. / 2016 / A “bag-of-arguments” Mechanism for Initial Verb Predictions / Language, Cognition and Neuroscience 31(5):577~596
Devlin, J. / 2018 / Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding
Federmeier, K. D / 1999 / A Rose by Any Other Name:Long-term Memory Structure and Sentence Processing / Journal of Mmory and Language 41(4):469~495
Chomsky, N. / 1957 / Syntactic Structures / Mouton
Chow, W. Y. / 2016 / A “bag-of-arguments” Mechanism for Initial Verb Predictions / Language, Cognition and Neuroscience 31(5):577~596
Devlin, J. / 2018 / Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding
Federmeier, K. D / 1999 / A Rose by Any Other Name:Long-term Memory Structure and Sentence Processing / Journal of Mmory and Language 41(4):469~495
함께 읽어보면 좋을 논문
논문 유사도에 따라 DBpia 가 추천하는 논문입니다. 함께 보면 좋을 연관 논문을 확인해보세요!
-
딥러닝 언어모델과 Surprisal을 활용한 언어분석
인공지능인문학연구
2023 .12
-
Evaluation of BERT-Based Models’ Ability to Linguistically Distinguish Between Human-Authored and Machine-Generated Texts
언어
2024 .12
-
코퍼스와 딥러닝 언어 모델을 활용한 문장 처리의 예측성과 행동 반응 시간과의 관계 연구
영어학
2020 .03
-
Decoding BERT’s Internal Processing of Garden-Path Structures through Attention Maps
영어학
2023 .01
-
BERT Learns More than Word Frequency Information: A Case Study of Do-Be Constructions
언어
2022 .09
-
Honorific agreement and plural copying revisited: Experimental and deep learning approach
언어연구
2022 .12
-
사전학습 언어모델의 기술 분석 연구
인문언어
2020 .01
-
최소대립 문장쌍을 활용한 한국어 사전학습모델의 통사 연구 활용 가능성 검증
언어와 정보
2021 .11
-
Is c-command Machine-learnable?
언어학
2021 .01
-
An Experimental Investigation of Discourse Expectations in Neural Language Models
영어학
2022 .01
-
한시 텍스트마이닝 기법의 적용과 한계
어문연구
2024 .09
-
중국인 고급 학습자의 한국어 격 표지 및 동사 의미 처리에 대한 뇌파 연구
이중언어학
2020 .01
-
Investigating Syntactic Interference Effects in Neural Language Models for Second Language Acquisition
영어학연구
2024 .02
-
Grammatical illusions in BERT: Attraction effects of subject-verb agreement and reflexive-antecedent dependencies
언어연구
2023 .06
-
Investigating Syntactic Transfer from English to Korean in Neural L2 Language Models
현대문법연구
2024 .03
-
Embedding Calculus with Nonword Properties Improves Word Sense Disambiguation
언어
2021 .06
-
DeepKLM - 통사 실험을 위한 전산 언어모델 라이브러리 -
언어사실과 관점
2021 .01
-
언어모델은 언어표현을 유연하게 처리하는가? - 재난문자를 활용한 평가 -
언어사실과 관점
2024 .05
-
딥러닝 기반 단어 임베딩을 적용한 사진 자막 영작문 채점 시스템
언어학
2021 .01
-
The Relationship between Language and Thought in Motion Event Cognition in Second Language Acquisition
어학연구
2019 .12
이 논문의 저자 정보
최근 본 자료
댓글(0)
0