Artificial Neural Systems: Principle and Practice

The aim of this chapter is to explain what a natural biological neuron is, and what an artificial neuron is. To this end, the first section introduces the biological neuron, explains its structure and its information transmission methods. The second section explains how an artificial neuron may be obtained from a corresponding biological neuron. The resources for the artificial neuron may be purely electrical in nature and the behaviour of the resulting electric circuit is expected to be similar to that of information transmission of a biological neuron.

The aim and objectives of this chapter is to present other types of artificial neuromorphic neurons with capability of reset and recovery. For this reason, the first section starts with the integrate-and-fire neuron, which has the propensity for reset. The second section introduces probability theory owing to the fact that many processes in the brain and central nervous system obey probability laws. The third section introduces another artificial neuromorphic neuron which employs a Poisson process and is closer in behaviour to a biological neuron.

The chapter’s aim and objectives are to provide an artificial neural-based fuzzy-logic foundation, and a general framework for design and analysis of ANN systems. The first section therefore introduces membership functions, define and give a relatively full operational description of fuzzy-logic neuron. Subsequent section two introduces ANN design principles and analysis from which a general wave neural network is derived. A full understanding of this chapter may be sufficient to design and analyse any artificial neural network system.

An algorithm is a sequence of operations that is used to find a solution to a problem. The sequence of the operations are often well organised so that it may be representable as a flowchart or state-machine when possible. The first few sections of chapter 4 illustrate this by way of clustering algorithm and nature-inspired algorithms. Having laid the fundamental background of artificial neural networks (ANN) in previous chapters, in terms of definitions, theorems, and equations, it is now possible to organise one or more of these elements in such a way that it provide intelligent solution to some problems. The organisation of the elements has led to an attempt to give a formal definition of ANN in this chapter. Important and notable organisation of concepts and definitions may both help in reformulating problems as well as providing solution for them. The suitability of these solutions may be accessed by placing one or more performance metrics on the corresponding ANN as shown in the last section of this chapter. Such is the theme of this chapter.

The aim and objectives of this chapter is to introduce the principles and fundamental concepts of elements which are used as building block of new and emerging neural systems. The first section introduces quantum logic and quantum algebra. Only those concepts relevant to ANNs’ gates production are introduced, and as such the mathematics involved has been kept to a minimum. The second section introduce a new non-volatile memory elements; the memristance. Since these phenomena are well known prior to their formal discovery (formulation), the fundamental concepts and the phenomena forms the subject of discussion in this chapter. The chapter has striven to be self-contained and brief in order to describe and impart information relevant to understanding of the ANNs’ elements.

The first chapter of part II of this book introduces various common learning algorithms. The aim of chapter 6 is to acquaint the readers with the present-day knowledge in learning paradigms. Filters may be employed in implementation of learning algorithms, and vice versa. As such, the first few sections introduce Adaptive Linear Neuron (ADALINE) and recursive Least-Square (RLS) algorithms. Artificial intelligent systems may possess functional characteristics of living biological brain. The multi-agent network and neuromorphic network introduced in subsequent sections are examples of ANN systems with functional characteristics of living biological brain. The ability of the brain to process data is unparalleled; the human research efforts have however been able to discover a close match in Bayesian networks such that more than half of this chapter is devoted to presenting various types of probability-density-based learning algorithms. This is followed, in conclusion, by a hybrid neuro-fuzzy neural network section. By reading this chapter, one may fully understand the common ANN systems, and thus easily implement an ANN if required.

This chapter describe various types of ANN systems in relative detail. It is the aim of the chapter to give descriptions of advanced ANN system in such a detail as to facilitate easy implementation. The first few sections are dedicated to the recent weightless neural networks. This is followed by a weighted neural system section. Two advanced Bayesian network are introduced subsequently. The last section of the chapter explains the dynamics of ANN and how ANN nay be evaluated. The chapter has given a relatively extensive description of typical advanced neural networks from various categories of ANN systems.

It is often required to select and combine two or more neural networks in order to process a given data. The aim and objectives of this chapter is to describe the selection and combination strategy of ANN systems. Two methods of ANNs’ selection and combination that are derived from principle are described in detail. Manual selection and combination which is possible only if it involve few networks are not considered. Also not considered are heuristically determined set of networks, because of additional large experimentation that must be performed to select a suitable number and configuration of ANNs. These hindrances are relieved by the selection and combination strategy described in this chapter. The chapter has described two methods of selection and combination of ANNs which may be applied to minimize ANN’s network errors. The selection and combination strategies descried in this chapter are principled, more robust, and of wider applicability than other alternatives.

Since Gaussian distribution may be employed as a universal approximator, it is clear that most modelling and optimisation problems could be solved by probabilitybased ANN systems. For this reason, chapter 9 concentrate on probability-based ANN systems. The first section introduces the random number generator, which has application in Markov-Chain and its hybrid, in subsequent sections. The fifth section describes the Restricted Boltzmann Machine (RBM) in detail. The Boltzmann machine may be a component network of Deep Belief Networks (DBN), which is described in the last section. The chapter has explained many algorithms related to DBN with great intuition, as this may facilitate better understanding and therefore implementation.

Considering the possible availability of non-volatile memory element, chapter 10 has presented a memristive neural network in its first section. Generally, the chapter aims to present those neural systems that, at present, exist as proof of concept, some of which are undergoing industrial tests. For this reason, quantum neural networks are described in relative detail, and without losing its generality. A category of Bayesian network commonly known as Deep Belief Networks (DBN) is revisited, this time to illustrate their industrial tests. This chapter has provided some informational resources on the present state of ANNs′ development. It has also striven to bridge the gap between researches, developments, and applications.

Two categories of ANN systems are able to model any intelligent Expert in great detail. These are Bayesian network, and neuromorphic network. Both are hampered by lack of adequate resources and lack of human knowledge. Research and development on these two categories of ANN systems is the subject of this chapter. The first section of chapter 11, on Bayesian network, specifically describes Hybrid Monte Carlo (HMC) and associated algorithms, after which areas of possible researches are highlighted. The second section is on neuromorphic network. It presents the current state of industrial development. The chapter has taken care to omit those conceptual developments which may not be achievable in near future. Illustration of the recent neuromorphic design has been given in concluding the second section. The chapter has provided research and development information resources on two advanced ANN systems.