Neural Networks in Computer and Cognative Science

Artificial neural networks (ANNs) attempt to replicate the human thought process by mimicking the basic building blocks of the brain—networks of neurons. This differs from traditional artificial intelligence approaches which tend to ignore the details of brain structure and instead focus on the “higher” patterns of thought such as symbol manipulation. While the ANN architecture has brought a number of contributions to computer science, its success in cognitive science is still debated.

ANNs are also known as neural networks, neuro-computing, neuromorphic systems, and parallel distributed computing amongst other descriptions. This general class of system design is often called connectionist architectures. Connectionism is the philosophical position that information in the brain is stored in the connections between neurons. According to this theory, there’s no single neuron which corresponds to, for example, ‘dogs’ in you mind. Instead, the idea of ‘dogs’ is stored in the strengths and types of connections between a multitude of individual neurons.

Neurons in neural networks are represented by small simple software units often called nodes or unsurprisingly ‘neurons.’ These units are connected together and each connection is given a ‘weight’ or value which signifies the strength of the connection. When a message is passed from one neuron to another the weight value either magnifies or reduces the signal strength. An individual neuron then uses a simple function to determine from all the available input messages whether or not to pass the signal on through its own output connections. Through the process of ‘back propagation’ results are sent back through the network informing the neurons to increase or decrease the weights depending on the success of the previous message passing events. In this way, neural networks can ‘learn’ which paths and connection weights produce successful results.

Neural networks have shown to be very good at a number of cognitive tasks most notably pattern recognition. Three well known examples include ANNs which synthesize text to speech (Sejnowski and Rosenberg (1987)), recognize and predict past tense English verbs (Rumelhart and McClelland (1986)), and predicting words in English sentences according to proper grammar (Elman(1991)). The Usenet comp.ai.neural-nets newsgroup FAQ lists ANN applications in a number of fields ranging from agriculture to criminology to gambling.

The video game industry has also made heavy use of ANNs. The recent game “Black & White” from Lionhead Studios while perhaps not the most gripping game in history certainly made headlines with its impressive use of artificial intelligence. In the game the user plays a deity in charge of small island of villages. To help in your endeavors you are given a giant ‘creature’ (a cow, ape, or tiger) which you can train as your avatar. The artificial intelligence of B&W’s creatures have become renown. In an interview given during the games release, developer Peter Molyneux described some of the surprising traits shown by the simulated beasts:

Molyneux says that during the game’s development, one of his creatures enjoyed playing games of catch with another, tossing rocks back and forth between them. After a while, it appeared that the other creature became tired of playing. So when Molyneux’s creature wasn’t looking, it placed a rock in a fire to make it hot and then kicked it with its feet onto Molyneux’s creature’s personal rock stash. “My creature picked up the hot rock,” he says, “and badly burnt its hands.” (Gamespy)

Another user relates this story:

“My creature is still pretty young – age 8, and I’m only in level 2 up to now. Anyways, while I was in level 1 I got the healing spell. So I thought, it would be pretty neat to teach my little ape this spell. Since Adiz wants to be always kind and generous to the people, he ran to the village to try his new spell. He looked around and I guess there was no one to heal. He got pretty upset about that. So he just picked up a guy and threw him as hard as he could against a mountain. The man rolled down, and – for some reason – survived. He was hurt pretty bad though, screamed like hell, and was trying to get back to his house as quick as possible. My ape of course saw that, and healed him. After that he looked at me all happy and smiling.” (Wired, 2002)

In fact, I’ve had my own experiences with the game that convinced me that my little ape had a temperament all of its own. Games like “Black & White” or “Creatures” have used neural network techniques to push the bounds of artificial intelligence. Indeed no one doubts the success neural networks have had in progressing the computer sciences. Additionally, as we’ve seen, connectionist architectures can perform a number of impressive cognitive tasks. Yet the question remains as to whether neural networks offer a fundamentally better model of human cognition as compared to more traditional artificial intelligence architectures.

Classical AI researchers argue that connectionism does not properly address rule based thought or ‘systematicity.’ For example, Marcus has argued against the Elman’s word predicting ANN that all that was really learned were associations not true abstract rules of grammar. This sort of rule or logic based reasoning can easily be built into classical symbol based systems. Yet for ANNs this sort of logic does not appear to be inherent.

Other researchers see connectionism as only an implementation detail—that it does not fundamentally enhance our understanding of the human thought process. These “implementationalists” seek a middle ground between classical AI researchers and strong connectionists. Their claim is that it doesn’t really matter if underneath the covers the artificial intelligence is an ANN or some rule base symbol crunching program as long as it possesses certain higher level cognitive features. To the implementationalists’ credit, while connectionists have shown ANNs to perform well in many aspects, they have yet to show that ANNs exhibit unique characteristics which will lead to breakthroughs in cognitive science.

Nevertheless, current trends in connectionist research continue to be promising. The most recent conference on “Computational Intelligence and Natural Computing” states that recent improvements have come in “(1) increasing the biological plausibility of network models (2) precise analysis of feed-forward network behavior with respect to traditional symbol-processing concepts (3) development and application of recurrent (feed-back) networks to sequence-processing tasks. (CINC 2003)”

I myself find connectionist architectures to be some of the best tools available in modeling brain behavior. The fundamental characteristics of learning, pattern recognition, robustness, distributed information processing, network architecture and emergent behavior in ANNs mirror much more closely our neurological understanding of the brain compared to the rigid rule based systems of “good old fashioned AI.” Hopefully as research in this area continues we will continue to see strides in both replicating cognitive behavior and in our understanding of the mind’s inner workings.

REFERENCES:

Computational Intelligence and Natural Computing Conference: http://axon.cs.byu.edu/CINC/ctnn.html

“Wild Things”, Wired Article on Black & White: http://www.wired.com/wired/archive/10.03/aigames.html

GameSpot Article on Black & White http://www.gamespot.com/gamespot/features/pc/hitech/p2_01.html

Neural Networks Warehouse Site http://neuralnetworks.ai-depot.com/

Neural Networks FAQ ftp://ftp.sas.com/pub/neural/FAQ.html

Standford Encyclopedia of Philosophy: Connectionism http://plato.stanford.edu/entries/connectionism/

St. Lewis University: Artificial Neural Networks Permalink Comments