DramaQA: Character-Centered Video Story Understanding with Hierarchical QA

Seongho Choi; Kyoung-Woon On; Yu-Jung Heo; Youwon Jang; Ahjeong Seo; Seungchan Lee; Minsu Lee; Byoung-Tak Zhang

doi:10.5626/KTCP.2021.27.1.7

DramaQA: Character-Centered Video Story Understanding with Hierarchical QA

Recommendations

Search

Questions

DramaQA: 계층적 질의응답과 함께하는 등장인물 중심 비디오 스토리 이해

Type: Academic journal

Author: Seongho Choi (서울대학교) Kyoung-Woon On (Kakao Brain) Yu-Jung Heo (서울대학교) Youwon Jang (서울대학교) Ahjeong Seo (서울대학교) Seungchan Lee (서울대학교) Minsu Lee (서울대학교) Byoung-Tak Zhang (서울대학교)

Journal: Korean Institute of Information Scientists and Engineers KIISE Transactions on Computing Practices Vol.27 No.1 KCI Accredited Journals

Published: 2021.1

Pages: 1 - 7 (7page)

DOI: 10.5626/KTCP.2021.27.1.7

Usage

📌

Topic

📖

Background

🔬

Method

🏆

Result

DramaQA: Character-Centered Video Story Understanding with Hierarchical QA

Research history (3)

2021

Video Story Understanding with Multi-level Character Attention Model

Seongho Choi , Kyoung-Woon On , Yu-Jung Heo and others 4 한국정보과학회 학술발표논문집 2021.06 Proceeding

DramaQA: Character-Centered Video Story Understanding with Hierarchical QA

Seongho Choi , Kyoung-Woon On , Yu-Jung Heo and others 5 KIISE Transactions on Computing Practices 2021.01 Academic journal

2019

DramaQA: Human Level Video Story Understanding through Multilevel Question-Answering

Seongho Choi , KyoungWoon On , Yu-Jung Heo and others 2 한국정보과학회 학술발표논문집 2019.12 Proceeding

Are you curious about the follow-up research of this article?
You can check more advanced research results through related academic papers or academic presentations.
Check the research history of this article

Abstract· Keywords

Report Errors

In this paper, we propose a novel video question answering (Video QA) task, DramaQA, for obtaining a comprehensive understanding of a video story. The DramaQA focuses on two perspectives: 1) hierarchical QAs as an evaluation metric based on the cognitive developmental stages of human intelligence, and 2) character-centered video annotations to model the local coherence of the story. Our dataset is built upon the TV drama “Another Miss Oh” and contains 16,191 QA pairs from 23,928 video clips of various lengths, with each QA pair belonging to one of four difficulty levels. We provide a total of 217,308 annotated images with rich character-centered visual annotations and coreference resolved scripts. In addition, we provide analyses of the dataset as well as a Dual Matching Multistream model which effectively learns character-centered representations of the video to answer questions about the video.

#비디오 질의응답 #비디오 스토리 이해 #질의응답 평가지표 #등장인물 중심 비디오 주석 #video question and answering #video story understanding #evaluation metric for QA #character-centered video annotation

요약
Abstract
1. 서론
2. DramaQA 데이터셋
3. Dual Matching Multistream 모델
4. 실험 결과
5. 결론
References

References (9)

Add References

Szilas, N., Interactive Drama on Computer: Beyond Linear Narrative, Proc. of the AAAI fall symposium on narrative intelligence, pp. 150-156, 1999.

Winston, P.H., The Strong Story Hypothesis and the Directed Perception Hypothesis, 2011 AAAI Fall Symposium Series, 2011.

Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A., Urtasun, R., Fidler, S., MovieQA: Understanding Stories in Movies through Question-Answering, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

Kim, K.M., Heo, M.O., Choi, S.H., Zhang, B.T., Deepstory: Video Story QA by Deep Embedded Memory Networks, IJCAI International Joint Conference on Artificial Intelligence, pp. 2016-2022, 2017.

Lei, J., Yu, L., Bansal, M., Berg, T.L., TVQA: Localized, Compositional Video Question Answering, EMNLP, 2018.

Recommendations

It is an article recommended by DBpia according to the article similarity. Check out the related articles!

Related Authors

최성호

Affiliation Univ. Leipzig

Fields Engineering > Computer Science Humanities > Religious Studies > Buddhist Studies

Articles 30 Usage 767

온경운

Affiliation 카카오브레인