지원사업
학술연구/단체지원/교육 등 연구자 활동을 지속하도록 DBpia가 지원하고 있어요.
커뮤니티
연구자들이 자신의 연구와 전문성을 널리 알리고, 새로운 협력의 기회를 만들 수 있는 네트워킹 공간이에요.
논문 기본 정보
- 자료유형
- 학위논문
- 저자정보
- 지도교수
- 임묘택
- 발행연도
- 2017
- 저작권
- 고려대학교 논문은 저작권에 의해 보호받습니다.
이용수0
이 논문의 연구 히스토리 (3)
- 이 논문의 후속연구가 궁금하신가요?
- 연관 학술논문 또는 학술발표를 통해 보다 발전된 연구결과를 확인하실 수 있습니다.
- 이 논문의 연구 히스토리 확인하기
초록· 키워드
오류제보하기
In this thesis a new 2-level recurrent neural network language model based on Continuous Bag of Words (CBOW) with applications to sentence classification is presented.
A neural network architecture for statistical language modeling (NNLM) has been shown to be competitive with and occasionally superior to the widely-used-n-gram language models. The vector representations of words learned by NNLM have been shown to carry semantic meanings and are useful in various natural language processing tasks. Especially, the CBOW which is advanced NNLM has attracted a great amount of attention in recent years. While standard NNLM predicts one target word given previous context words, the CBOW predicts one target word given surrounding context words and it is superior to the standard NNLM. However, disadvantage of the CBOW is that it only considers fixed length of context because the basic structure of the CBOW is neural network which has fixed length of input.
In contrast, recurrent neural network based language model (RNNLM) does not use limited size of context, but it only considers previous context words. Therefore the advantage and disadvantage of the RNNLM is complementary to those of the CBOW. To apply the CBOW to RNNLM, bidirectional recurrent neural network (BRNN) is adopted. Furthermore, 2-level recurrent neural network (RNN) structure is proposed to abstract the context representation that have proved so effective in deep networks.
The proposed model herein encode many linguistic patterns and improve upon sentiment analysis and question classification benchmarks.
A neural network architecture for statistical language modeling (NNLM) has been shown to be competitive with and occasionally superior to the widely-used-n-gram language models. The vector representations of words learned by NNLM have been shown to carry semantic meanings and are useful in various natural language processing tasks. Especially, the CBOW which is advanced NNLM has attracted a great amount of attention in recent years. While standard NNLM predicts one target word given previous context words, the CBOW predicts one target word given surrounding context words and it is superior to the standard NNLM. However, disadvantage of the CBOW is that it only considers fixed length of context because the basic structure of the CBOW is neural network which has fixed length of input.
In contrast, recurrent neural network based language model (RNNLM) does not use limited size of context, but it only considers previous context words. Therefore the advantage and disadvantage of the RNNLM is complementary to those of the CBOW. To apply the CBOW to RNNLM, bidirectional recurrent neural network (BRNN) is adopted. Furthermore, 2-level recurrent neural network (RNN) structure is proposed to abstract the context representation that have proved so effective in deep networks.
The proposed model herein encode many linguistic patterns and improve upon sentiment analysis and question classification benchmarks.
목차
제 1 장 서론 11.1 연구의 배경 11.2 연구의 목적 및 내용 41.3 논문의 구성 5제 2 장 배경 이론 72.1 순환 신경망 72.1.1 순환 신경망의 구조 82.1.2 Back-Propagation Through Time 92.1.3 기울기 소실 문제 122.2 언어 모델 132.2.1 신경망 언어 모델 142.2.2 Continuous Bag of Words 172.2.3 순환 신경망 언어 모델 19제 3 장 CBOW 기반 2-레벨 순환 신경망 213.1 CBOW 기반 2-레벨 순환 신경망의 구조 213.2 순환 신경망 유닛 233.2.1 Long Short Term Memory 243.2.2 Gated Recurrent Unit 273.3 Regularization 30제 4 장 실험 결과 및 분석 334.1 실험 환경 및 구성 334.2 언어 모델링 실험 364.2.1 순환 신경망 유닛의 모델링 정확도 비교 374.2.2 언어 모델의 모델링 정확도 비교 394.3 문장 분류 실험 43제 5 장 결론 46참고 문헌 49
최근 본 자료
댓글(0)
0