메뉴 건너뛰기
Library Notice
Institutional Access
If you certify, you can access the articles for free.
Check out your institutions.
ex)Hankuk University, Nuri Motors
Log in Register Help KOR
Subject
Humanities
General Humanities History Philosophy Religious Studies Linguistics and Literature Other Humanities
Social science
General Social Science Business Management Tourism Education Military Science Library and Information Science Law Sociology Journalism and Media Studies Psychological Science Political Science and Diplomacy Geography Public Administration Economics International Trade Anthropology Other Social Science
Natural science
General Natural Science Physics Biology Life Sciences Mathematics/Statistics Astronomy Earth Science Chemistry Other Natural Science
Engineering
General Engineering Architectural Engineering Mechanical Engineering Industrial Engineering Material/Energy Engineering Electrical and Electronic Engineering Marine and Ocean Engineering Chemical Engineering Computer Science Other Engineering
Medical science
General Medical Science Nursing Science Veterinary Pharmacy Traditional Korean Medicine Basic Medicine Clinical Medicine Other Medicine
Agriculture and Fishery
General Agriculture and Fishery Agriculture Fishery Science Food Science Forestry Animal Husbandry Marine Transportation Studies Other Agriculture and Fishery
Art and Kinesiology
General Arts Design Fine Arts Cosmetology Photography Music Fashion Kinesiology Performing Media Arts Other Arts and Kinesiology
Interdisciplinary Studies
Interdisciplinary Research Science and Technology Studies Brain and Cognitive Science Technology Policy Other Interdisciplinary Studies

KOR

Evaluating of Korean Machine Reading Comprehension Generalization Performance via Cross-, Blind and Open-Domain QA Dataset Assessment
Recommendations
Search
Questions

기계독해 말뭉치의 교차 평가, 블라인드 평가 및 오픈도메인 질의응답 환경 평가를 통한 한국어 기계독해의 일반화 성능 평가

논문 기본 정보

Type
Academic journal
Author
Joon-Ho Lim (한국전자통신연구원) Hyun-ki Kim (한국전자통신연구원)
Journal
Korean Institute of Information Scientists and Engineers Journal of KIISE Vol.48 No.3 KCI Excellent Accredited Journal
Published
2021.3
Pages
275 - 283 (9page)
DOI
10.5626/JOK.2021.48.3.275

Usage

cover
📌
Topic
📖
Background
🔬
Method
🏆
Result
Evaluating of Korean Machine Reading Comprehension Generalization Performance via Cross-, Blind and Open-Domain QA Dataset Assessment
Ask AI
Recommendations
Search
Questions

Abstract· Keywords

Report Errors
Machine reading comprehension (MRC) entails identification of the correct answer in a paragraph when a natural language question and paragraph are provided. Recently, fine-tuning based on a pre-trained language model yields the best performance. In this study, we evaluated the ability of machine-reading comprehension method to generalize question and paragraph pairs, rather than similar training sets. Towards this end, the cross-evaluation between datasets and blind evaluation was performed. The results showed a correlation between generalization performance and datasets such as answer length and overlap ratio between question and paragraph. As a result of blind evaluation, the evaluation dataset with the long answer and low lexical overlap between the questions and paragraphs resulted in less than 80% performance. Finally, the generalized performance of the MRC model under the open domain QA environment was evaluated, and the performance of the MRC using the searched paragraph was found to be degraded. According to the MRC task characteristics, the difficulty and differences in generalization performance depend on the relationship between the question and the answer, suggesting the need for analysis of different evaluation sets.
#기계독해 #KorBERT #사전학습 언어모델 #일반화 평가 #machine reading comprehension #pre-trained language model #mrc generalization

Contents

요약
Abstract
1. 서론
2. 관련 연구
3. 기계독해 말뭉치의 교차평가를 통한 일반화 성능 평가
4. 블라인드 평가를 통한 일반화 성능 평가
5. 오픈-도메인 질의응답 환경에서의 기계독해 일반화 성능 평가
6. 결론
References

References (15)

Add References
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang, "Squad: 100,000+ questions for machine comprehension of text," Proc. of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, 2016. google schola Pranav Rajpurkar, Robin Jia, and Percy Liang, "Know what you don’t know: Unanswerable questions for squad," arXiv preprint arXiv:1806.03822, 2018. https://rajpurkar.github.io/SQuAD-explorer/ https://korquad.github.io/category/1.0_KOR.html Chih Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng, and Sam Tsai, "Drcd: a chinese machine reading comprehension dataset," arXiv preprint arXiv: 1806.00920, 2018.

Recommendations

It is an article recommended by DBpia according to the article similarity. Check out the related articles!

Related Authors

Frequently Viewed Together

Recognizing Page Title and Infobox Attribute for Answer Extraction from Wikipedia Infobox
Jeong Heo ,  Pum Mo Ryu ,  Hyun Ki Kim and others 3 Journal of KISS : Software and Applications 2013 .09
Morphological Awareness on Derivational Affixes in Accordance With Vocabulary Knowledge in School-Age Children
Bhu Ja Chung ,  Young Tae Kim Journal of speech-language & hearing disorders 2020 .07
S-VQA: Specific Visual Question Answering system exploited the image tag
Young-jae An ,  Dong-ho Lee 한국정보과학회 학술발표논문집 2018 .06
Question Taxonomy of Design Management Problems for a Natural Language Search System – Focused on Korean Small and Medium Enterprises
Kim ,  Bo-Ra ,  Jeong and others 5 KSDS Conference Proceeding 2017 .11
A Method for Question Type Extractions and Query Transformations at Question Answering System
Kyoung-Min Lim ,  Kyu-Sang Kim ,  Dong-Joo Kim and others 1 대한전자공학회 학술대회 2006 .11

Recently viewed articles

The Negative Effect of a Smiling Model in Luxury Brand’s Advertising : Focus on the Mediating Effect of Perceived Competence of the Brand
Kim, Tae Min, Yoo 외 1명The Research Institute of Social Science2023.03
 Korean Spacing Module using Eojeol dictionary and Morpheme dictionary
Chanjun Park, Jin-hyuk Choi, Pummo RyuKorean Institute of Information Scientists and Engineers2018.12
 Physical, Psychological and Social Benefits of Participation in Youth Sports for Children Ages 9-12 Years
Illa R. Haley, Seung Ho ChangKorean Alliance For Health, Physical Education, Recreation, And Dance2021.08

Comments(0)

0

Write first comments.