In this thesis, Fuzzy Clustering-based Context Neural Networks(FCCNNs) prediction model and Fuzzy Clustering-based Softmax Neural Networks(FCSNNs) pattern classifier are proposed respectively. The objective of this study is focused on the development of the design methodologies for redesigning the network structure and enhancing the learning methods of fuzzy clustering-based neural networks. While the proposed FCCNNs prediction model includes the new network structure called context layer for reasoning the membership values of output space formed by FCM clustering, (b) the proposed FCSNNs pattern classifier takes into consideration a cross entropy error function for improving learning method and L2 regularization in the pattern classifier is used to reduce overfitting as well as to enhance generalization ability.
(a) First, the key features of the proposed Fuzzy Clustering-based Context Neural Network prediction model can be summarized as follows:
1) The connection weights between the input layer and the hidden layer of networks are given as single constant term and are used as the prototypes of fuzzy c-means(FCM) clustering. The connection weights can be adjusted through the learning mechanism of fuzzy c-means clustering and then the membership(partition) values of FCM clustering are used as the output of hidden layer of the networks.
2) The context layer between the hidden layer and the output layer of the networks is additionally considered to approximate the context space of output variable. The outputs of the context layer are given as the approximate values obtained from the membership values of output variable used as the input of FCM clustering and those approximate values are depicted by the linear combination of both connection weights as linear function and the outputs of the hidden layer. The connection weights are updated by linear least squares method(LLSM)-based learning.
3) In the output layer, the model output is defined by the linear combination of both connection weights and the output of the context layer. The connection weights are given as linear function and adjusted by LLSM.
4)The Approximation ability as well as the generalization ability of the proposed FCCNNs prediction model is remarkably enhanced when compared with the conventional fuzzy clustering neural networks using linear weighted least squares method(LWLSM) or linear least squares method(LLSM).
(b) Secondly, the key points of the proposed Fuzzy Clustering-based Softmax Neural Network(FCSNNs) pattern classifier can be enumerated as follows:
1) In the proposed pattern classifier, the cross entropy error function is used as cost function instead of using the sum of squared error(SSE) and softmax function is exploited in the nodes of the output layer to normalize model output between 0 and 1.
2) The learning mechanism for adjusting the connection weights of the hidden layer is the same as the proposed FCCNNs prediction model explained previously, but the connection weights linked to the output layer are modified by nonlinear least squares method based on Newton’ method-based learning.
3) L2 regularization is considered to avoid the degradation of generalization ability caused from overfitting. By adding L2 penalty term to the cross entropy error function, the performance of the proposed FCSNNs pattern classifier outperforms that of previous classifiers reported in literatures.
From the viewpoint of performance improvement through the novel network structure as well as learning method, the proposed design methodologies for prediction model and pattern classifier are discussed and analyzed with the aid of a diversity of data sets such as some machine learning data sets, Yale and AT&T face data sets, and universal format radar data