I have written several articles on Artificial Neural Networks earlier but they were just random articles on random concepts. This series of articles will give you a detailed idea about Artificial neural networks and concepts related to it. The resources and references to all the contents will be mentioned at the end of series so you can study all concepts in depth.
So, let’s start with a very basic question what is AI and what are artificial neural networks? In this very first article in this series I will try to answer this basic questions and then we will go ahead in depth in further articles.
What is Artificial Intelligence?
AI has been described as software that behaves in some limited ways like a human being. The word artificial comes from the Latin root word facere arte which means “make something” thus AI translates loosely to man-made intelligence. AI has been defined in many ways. Winston [1984] suggests one definition of AI as the study of ideas that enable computers to be intelligent. Rich and Knight [1991] define AI as the study of how to make computers do things which, at the moment, people do better.
The following are some more common definitions and/or descriptions of AI:
- AI is intelligent because it learns;
- AI transforms data into knowledge;
- AI is about intelligent problem solving;
- AI embodies the ability to adapt to the environment, to cope with incomplete or incorrect knowledge.
While artificial intelligence techniques have only recently been introduced in finance, they have a long history of application in other fields. Experience to date across a wide range of non-financial applications has been mixed. Patrick Winston, a leading AI researcher and the head of MIT’s AI Laboratory conceded that the traditional AI methods such as search methods, predicate calculus, rule-based expert systems, game-playing, etc. have achieved little progress [Gallant 1994]. The problem domain that traditional AI methods seem to fail in is in the trivial and common sense-type of tasks that humans find easy such as recognizing faces and object, walking, etc.
Therefore, it was natural for AI researchers to turn to nature and the physical laws and processes for inspiration to find better solutions. As a result, many of the contemporary artificial intelligence tools developed in the natural sciences and engineering field have successfully found their way into the commercial world. These include wavelet transformations and finite impulse response filters (FIR) from the signal processing/electrical engineering field, genetic algorithms and artificial neural networks from the biological sciences, chaos theory and simulated annealing from the physical sciences. These revolutionary techniques fall under the AI field as they represent ideas that seem to emulate intelligence in their approach to solving commercial problems. All these AI tools have a common thread in that they attempt to solve problems such as the forecasting and explanation of financial markets data by applying physical laws and processes. Pal and Srimani [1996] state that these novel modes of computation are collectively known as soft computing as they have the unique characteristic of being able to exploit the tolerance imprecision and uncertainty in real world problems to achieve tractability, robustness, and low cost. They further state that soft computing are often used to find approximate solution to a precisely (or imprecisely) formulated problem. Huffman of Motorola states that “At Motorola, we call neural networks, fuzzy logic, genetic algorithms and their ilk natural computing” [1994].
These contemporary tools are often used in combination with one another as well as with more traditional AI methods such as expert systems in order to obtain better solutions. These new systems that combine one or more AI methods (which may include traditional methods) are known as ‘hybrid systems’. An example of a hybrid system is the financial trading system described in Tan [1993] which combines an artificial neural network with a rule-based expert system. Lawrence [1994] preferred to use the term computer intelligence to describe expert systems and artificial neural networks as she felt it was less misleading and less controversial in defining the “intelligence” emulated by such systems.
What are Artificial Neural Networks?
Artificial Neural Network (ANN) models were inspired by the biological sciences which study how the neuroanatomy of living animals have developed in solving problems. According to Nelson and Illingworth [1990], ANNs are also called:
- Parallel distributed processing models
- Connectivist/connectionism models
- Adaptive systems
- Self-organizing systems
- Neurocomputing
- Neuromorphic systems
ANNs consist of many interconnected processors known as neurons that perform summing function. Information is stored in the weights on the connections. We will have more detailed discussions on technical aspects of neural networks in further articles.
An ANN mimics the human brain’s biological neural network. The biological neural network is the mechanism through which a living organism’s nervous system functions, enabling complex tasks to be performed instinctively. The central processing unit of that nervous system is known as a “neuron”. The human brain has around 10 to 100 billion neurons, each connected to many others by “synapses”. The human brain has around 100 trillion synapses. These connections control the human body and its thought processes. In short, they attempt to replicate the learning processes of the human brain.
The first ANN theories were expounded by researchers attempting to explain human behavior and the thinking process by modeling the human brain. To this day, many of the prominent researchers in the ANN field consist of researchers with background in psychology.
The four distinct areas of research in ANNs are:
- Using ANNs to model the biological networks in order to gain understanding of the human brain and its functions. This area is of particular interest to psychologists and researchers in neuroanatomy;
- Using ANNs as an educational tool in order to gain understanding on how to solve complex tasks that traditional AI methodologies and computer algorithms have had difficulty in solving. Researchers in this area include computer scientists, engineers, etc., who are mainly interested in constructing better computer algorithms by studying the problem-solving process of an ANN;
- Using ANNs to solve real world-types of problems in various commercial applications. Many researchers in this area have backgrounds in areas other than those related to ANN. The attraction of using an ANN is the simplicity in using it as a tool and the reported ANN-based commercial application successes. There are many ANN software packages that are user-friendly enough for new users to start using without requiring them to have an in depth knowledge of the ANN algorithms. This is unlike conventional computer techniques which require a user to thoroughly understand the algorithm before writing program to apply it. In the case of ANNs, all a user needs to know is how to present the problem at hand in a form that an ANN can understand; and
- Improving ANN algorithms. Researchers in this field are interested in constructing better ANN algorithms that can ‘learn’ or model more efficiently, i.e. quicker training times and/or more accurate results.
Research efforts on ANNs are being conducted on a global basis. Nelson and Illingworth [1991] state that Jasper Lupo, the deputy director of the Tactical Technology Office of the Defense Advanced Research Projects Agency (DARPA), called the neural network technology “more important than the atom bomb” [Johnson and Schwartz 1988]. According to Nelson and Illingworth, DARPA originally earmarked US$390 million for an eight-year neural network program but even when the original funding was reduced to US$33 million over 17 months, there were still many applications for the research grants. More recently, Turban and Trippin [1996], state that] following the five-year research program, the Department of Defense (D.O.D) is planning to spend an additional US$15 million in neural network research over the period 1995–2000. They further claim that the Japanese have embarked on a 10-year, US$20 million program to further develop neural network technology, mainly in the commercial arena.
Japan’s main ANNs research is sponsored by its government under its post-fifth generation computer program called “The Human Frontiers”. However, Japanese corporations are already developing products based on ANNs. Examples of Japanese corporation’s involvement with ANN technology are:
- Sharp Corporation’s optical character reading of printed Japanese program [Shandle 1993],
- Nippon Steel’s casting breakthrough prediction program [Shandle 1993],
- Hitachi’s ANN hardware system design [Shandle 1993],
- Ricoh’s experimental neurocomputer that runs without software and acquires all computing capabilities through learning [Dambrot 1992],
- Fujitsu’s ANNs-based mobile robot controller, and NEC Corporation’s neurocomputer [Nelson and Illingworth 1991], etc.
Europe’s ANNs research effort is called ESPIRIT II and is a five year project involving eight countries and several hundred worker-years of effort [Mehta 1988]. This has been supplemented by a new program announced by ESPIRIT in early 1989, known as the Application of Neural Networks for the Industry (ANNIE) [Newquist III, 1989]. Nelson and Illingworth [1991] states the following about ANNs research effort in individual European countries:
- Germany has a US$250 million 5 year program [Johnson 1989b];
- France, probably has the most active development with six neural-based microchip projects in Paris alone [Johnson 1988];
- Netherlands research has moved from independent research to government sponsored and coordinated research; and
- United Kingdom has a US$470 million project.
The UK Advisory Council for Science and Technology forecasted the market for neural network products in 1997 at US$1 billion which resulted in the UK Department of Trade and Industry (DTI) announcement of a Technology Transfer program that will invest 5.7 million pounds over the next three years to raise awareness of the benefits of neural networks to 6,000 UK companies [Milton 1993].
Conclusion:
This was just a short introduction of AI and Artificial Neural Networks with description of how they are impacting world on a great scale. I next article we will see some applications of artificial neural networks in various industries.
