Software versions of artificial neural networks (ANN) which are the only available implementations so far lack the inherent parallelism provided by neural networks. The realization of neural learning in fast, compact and reliable hardware is a tough task. A major bottleneck in analog neural network hardware is the implementation of nonvolatile programmable weight storage. The size of analog multipliers for the weight storage is well over practical implementations. Another problem in hardware realization of neural network is difficulty in implementing efficient learning rules like back-propagation (BP).
In this study, the implementation of the memristor-based multilayer neural networks and the simplest hardware-based learning architecture called RWC(Random Weight Change) is aimed. In this end, a higher priority is imposed on the effective implementation of neural synapses (weights). Two major functions which should be embeded in synapses are programmability of memrory and analog multiplication. A recent physical realization of “memristor”, a non-volatile programmable resistor, has opened up new possibilities. Memristor is a nonvolatile memory which is programmable. Also, analog multiplication which is essential in synaptic processing is embeded in a memristor. However, commercially available memristor is still not seen on the market. A memristor emulator which can imitate a TiO2 memristor precisely is designed with circuits in this paper.
Though memristor has very excellent features to implement neural synapses, the full implementation of neural networks is not possible only with a single memristor since negative and zero weighting as well as positive weighting are also necessary in neural synases. We have also proposed an universal synaptic architecture combining 4 memristors called memristor bridge synapse which can implement all the signs of weighting (positive, negative and zero). Multilayer neural networks can be built via the connections of multiple neurons with synapses. An active load-based circuit of neuron cell body is employed to integrate the weighted inputs of a node.
To work around difficulties in implementing efficient learning rule like BP in hardware, the circuit design of Random Weight Change (RWC) algorithm is also proposed. The weight of each synapse is changed randomly by a fixed amount in each iteration and is easy to implement. To show a successful learning with our designed neural network circuits, a learning example has been demonstrated about XOR problem.
Though circuit-based RWC algorithm requires more iterations than the software-based backpropagation algorithm, we show that the hardware-based learning of RWC is two orders faster than its software counterpart. The proposed learning scheme has been designed based on a real Die Chip design rule.