The question 'What is a neural network?' is ill-posed.
            
            
            
            
            
             - Pinkus
   (1999) 

First of all, when we are talking about a neural network, we should more
properly say "artificial neural network" (ANN), because that is what we mean
most of the time in comp.ai.neural-nets. Biological neural networks are much
more complicated than the mathematical models we use for ANNs. But it is
customary to be lazy and drop the "A" or the "artificial". 

There is no universally accepted definition of an NN. But perhaps most
people in the field would agree that an NN is a network of many simple
processors ("units"), each possibly having a small amount of local memory.
The units are connected by communication channels ("connections") which
usually carry numeric (as opposed to symbolic) data, encoded by any of
various means. The units operate only on their local data and on the inputs
they receive via the connections. The restriction to local operations is
often relaxed during training. 

Some NNs are models of biological neural networks and some are not, but
historically, much of the inspiration for the field of NNs came from the
desire to produce artificial systems capable of sophisticated, perhaps
"intelligent", computations similar to those that the human brain routinely
performs, and thereby possibly to enhance our understanding of the human
brain. 

Most NNs have some sort of "training" rule whereby the weights of
connections are adjusted on the basis of data. In other words, NNs "learn"
from examples, as children learn to distinguish dogs from cats based on
examples of dogs and cats. If trained carefully, NNs may exhibit some
capability for generalization beyond the training data, that is, to produce
approximately correct results for new cases that were not used for training.

NNs normally have great potential for parallelism, since the computations of
the components are largely independent of each other. Some people regard
massive parallelism and high connectivity to be defining characteristics of
NNs, but such requirements rule out various simple models, such as simple
linear regression (a minimal feedforward net with only two units plus bias),
which are usefully regarded as special cases of NNs. 

Here is a sampling of definitions from the books on the FAQ maintainer's
shelf. None will please everyone. Perhaps for that reason many NN textbooks
do not explicitly define neural networks. 

According to the DARPA Neural Network Study (1988, AFCEA International
Press, p. 60): 

   ... a neural network is a system composed of many simple processing
   elements operating in parallel whose function is determined by
   network structure, connection strengths, and the processing performed
   at computing elements or nodes. 

According to Haykin (1994), p. 2: 

   A neural network is a massively parallel distributed processor that
   has a natural propensity for storing experiential knowledge and
   making it available for use. It resembles the brain in two respects: 

   1. Knowledge is acquired by the network through a learning process. 
   2. Interneuron connection strengths known as synaptic weights are
      used to store the knowledge. 

According to Nigrin (1993), p. 11: 

   A neural network is a circuit composed of a very large number of
   simple processing elements that are neurally based. Each element
   operates only on local information. Furthermore each element operates
   asynchronously; thus there is no overall system clock. 

According to Zurada (1992), p. xv: 

   Artificial neural systems, or neural networks, are physical cellular
   systems which can acquire, store, and utilize experiential knowledge.

References: 

   Pinkus, A. (1999), "Approximation theory of the MLP model in neural
   networks," Acta Numerica, 8, 143-196. 

   Haykin, S. (1994), Neural Networks: A Comprehensive Foundation, NY:
   Macmillan. 

   Nigrin, A. (1993), Neural Networks for Pattern Recognition, Cambridge,
   MA: The MIT Press. 

   Zurada, J.M. (1992), Introduction To Artificial Neural Systems, Boston:
   PWS Publishing Company.