Applications of adaptive systems
Asking what you can do with adaptive systems such as neural networks is a bit like asking what you can do with computer programming. The answer is the same: more or less anything that deals with information. There are however certain standard problems and problem types that adaptive systems are applied to. This article gives an overview of typical problems as well as some practical examples.
Types of problems
Problems solved with function modeling are those where you wish to determine numeric outputs given a set of numeric inputs. The adaptive system builds a model that finds the relationship between the inputs and the wanted output. This covers a very wide range of problems. An example would be estimating house prices (output) given a number of metrics as input (house size, land size, distance to nearest school....).
Another example would be training an adaptive system to drive a car where the input is a picture of the road ahead and the outputs are the control for the steering wheel, throttle etc
function modeling covers an extremely wide area of applications and is the most common thing adaptive systems are used for. You can see it as an alternative to conventional programming. In the traditional way, you code line by line, instruction by instruction how the computer should perform a calculation or task. With function modeling, instead of telling the computer how to do something, you show it examples of what you want to be done. You provide it with data on inputs and show it what outputs you want and the adaptive system builds the actual function autonomously.
Classification is a special case of function modeling which deals specifically with pattern recognition. The adaptive system learns how to categorize the input it receives into a set of classes. Adaptive systems have shown superiority to conventional methods in all aspects of classification and are very popular in the field of pattern recognition.
A typical example of classification would be handwriting recognition. The input to such a system is an image of the drawn character and the output tells which character it was (as text or a number).
Another example would be beer quality testing (actually in use in some breweries) where an adaptive system sorts brewing batches into different quality classes (output) based on various chemical readings (input).
Classification also follows the learning-by-example paradigm. The adaptive system is trained with example inputs and their classes. From that the adaptive system builds a general classifier that can then be used on new input data.
The problems described above all deal with static data - i.e. where one specific input always should lead to one specific output. In systems that have memory (where current outputs depend on past inputs) and internal dynamics a different type of problem emerges. Such problems involve temporal dynamics and are far more complex to solve. Fortunately Synapse makes it just as easy as using any other adaptive system.
A typical dynamic filtering problem is for instance adaptive noise reduction which can be used to very efficiently remove noise from for instance audio signals.
Another example of a dynamic filtering problem is for instance classification of signal dynamics. Unlike regular classification that maps a set of inputs to a class, its dynamic counterpart classifies the way a signal changes over time. One such example is body-movement recognition, where a person is uniquely identified based on the way that individual moves.
Dynamic filtering operates as well on a learning-by-example principle. The difference is that the examples here are entire sequences of inputs.
Prediction is a special type of dynamic filtering where past values are used to predict future ones. It is commonly used in forecasting where one has a time series that extends to the present and one wants to predict the time series for future samples.
One typical example of forecasting is for instance predicting the behavior of the stock market or forex market. Another example of prediction is anticipating spikes in the electrical grid.
Predicting the future output of very complex systems is a difficult task. Adaptive systems however have shown themselves, trained on the right data, quite capable of producing good predictions. They are consistently better than more traditional methods.
System Identification is another special type of dynamic filtering where the aim is to build a mathematical model that captures the functions and dynamics of a real-world system. Such models are very important for analysis, simulation, prediction, monitoring and control system design. The biggest difference between adaptive systems and traditional system identification methods are that adaptive systems such as neural networks are non-linear, giving them an order of magnitude more modeling power.
Another big advantage is that adaptive systems can use passive probing to create a model. Conventional methods usually require that you explicitly capture the spectral properties of the system by sending white noise through it. One might observe that, you really don't want to send white noise through a nuclear power plant. Adaptive systems on the other can "listen" in on the normal signals and still build good general models - no need for intrusive probing.
A typical example of adaptive systems identification can be found in process industry. There it is important to have a good model of the plant (for instance a chemical plant) in order to detect anomalies, perform optimization etc
The result can be reversed as well, forming an inverse system identification - more commonly known as a control system. In such a mode you tell the adaptive system what output you want from the plant, and it will tell it what inputs you have to provide.
Clustering refers to grouping data by similarity. Adaptive systems can perform autonomous clustering on data - i.e. you do not have to provide any type of feedback signal or tell it what to do.
A common application of clustering is market segmentation. Through basic data from a survey, a clustering adaptive system can group the people according to similarity (multi-dimensional). Clustering adaptive systems are very useful tools for data mining. Synapse uses several types of such clustering methods for data visualization and pre-processing.
Feature Extraction is the problem of finding characteristic traits in data. Adaptive systems can perform autonomous feature extraction from data.
This feature is very practical for data compression, and often used as a pre-processing stage into classifiers of function modelers. Since adaptive systems adapt to the specific data, they can do far better compression than one-method-fits-all generic compression schemes.
Adaptive systems can be trained to learn the behavior of a system (or person) and can react when that behavior becomes uncharacteristic. This is called anomaly detection or novelty filtering.
A typical example would be network security where the network traffic of a computer is monitored and reported to an adaptive system. The adaptive system then over time builds up a model of what constitutes normal operation. When something out of the ordinary (such as a hacking attempt) occurs, the adaptive system reports it.
Anomaly detection is also important in industrial plants as well as medical monitoring equipment. They can also be used as financial market indicators.
- Financial forecasting – Adaptive systems are very popular tools for non linear prediction of financial data such as stock market forecasting. They have proven themselves to be very powerful tools superior to all traditional methods. All larger stock broker companies have various forms of adaptive systems. It is only recently however that software at reasonable price levels (targeting private investors, day traders etc) have entered the market.
- Fraud detection – Banks and other financial institutes use adaptive systems for fraud detection (credit cards, transactions etc). The adaptive system learns the habits and style of the account holder and it can detect anomalies. Since the broader introduction of these technologies, credit card fraud has dropped, breaking a decade long trend of ever increasing rates.
- Market segmentation – Clustering neural networks can be used to autonomously do market segmentation. This enables large amount of data to be analyzed without any human interference and works good in combination with automated data collection (as is more popular with internet surveys etc).
- Credit ratings – Adaptive systems are used in many loan approval and credit rating systems that evaluate the risk of giving out a loan based on data about the customer. They can make a very good risk analysis and generally outperform methods such as expert systems as they draw their conclusions from actual data, rather than from an idealized model.
- Real estate evaluation – Adaptive systems often perform better than human real estate agents in estimating house and property values. The systems are trained with historic data for evaluated objects, learning to predict the actual final sales price. Also, through sensitivity analysis it is possible to get qualitative information about the various preferences of the house buyers.
- Human resources – One of the more controversial applications of adaptive systems is in employee evaluation and when screening job-applicants. Adaptive systems have shown to be quite good at predicting future job performance based on biographical data.
- Plant modeling and control – One of the most successful and popular applications of adaptive systems is plant modeling and control in process industry. Adaptive systems provide a much cheaper, more reliable and robust alternative to traditional physical modeling of plants. Successful applications have been made for many process industries (paper, chemicals etc).
- Electrical load balancing – State of the art load balancing systems use adaptive systems for optimization and regularization.
- Robotics – Robot control was one of the first areas where adaptive systems were used. Systems such as neural networks can be used for many purposes ranging from mechanical control, vision to calibration.
- Automotive industry – Modern day cars use a multitude of different embedded systems ranging from traction control to controllers for seat settings. In just about every of those systems, adaptive systems have a significant advantage over traditional programming. Recently Aston Martin integrated a neural network system into the engine of their DB9 model. The neural network makes sure that the V-12 engine never misfires, something conventional software could not handle.
- Washing machines – Most modern day washing machines include circuitry for both fuzzy logic as well as a special type of neural network that is used to optimize the water flow, the amount of detergent etc relative to various sensor readings of the clothes inside the machine.
- Optical mice – Optical mice use a special type of pre-trained adaptive systems to detect movement. The adaptive system used is a type of novelty filter that detects anomalies relative an equilibrium (input data are images from the optical mouse CCD)
- Toasters – In Japan you cannot buy a toaster today that has not got software or circuitry using fuzzy logic and neural nets to control the toasting process based on sensor readings of the bread inserted.
- Inkjet printers – Inkjet printers use neural nets to adjust the ink flow on the printer head based on paper type, print type etc.
- Microwave ovens – Adaptive systems are used in more advanced microwave ovens to regulate the power output based on various sensor readings of the food.
- Digital cameras – Digital cameras use adaptive systems software to adjust various parameters depending on the environmental conditions.
- Medical diagnosis – Adaptive systems have been used to diagnose cancer and other diseases. Neural networks have been successful in making very accurate diagnostics on breast cancer. The inputs used are measurements form cell samples. Other diseases have been successfully and accurately diagnosed using adaptive systems. These include liver damage, diabetes and heart conditions.
- Bio informatics – Adaptive systems such as kernel machines have been successfully used in the human genome project (HUGO) for classification of DNA base pairs. Adaptive methods consistently outperform traditional search algorithms.
- Drug interaction prediction – Adaptive systems can be used to predict side effects of the interaction of various drugs. In the same way neural nets can be used to understand allergic reactions and their causes (there are often several substances interacting, together causing the reaction)
- Survival probability estimation – Adaptive systems can be used to perform triage based on patient data. They give the benefit of high accuracy while at the same time relieving the doctors from making the difficult decision of choosing who is to be treated or who gets a transplant. In the UK there is a national experimental program that uses neural networks to determine who gets liver transplants based on 80 different measured factors.
- Security – Adaptive systems are very popular in intrusion detection systems that learn to distinguish between regular network traffic and malicious intrusion attempts. The adaptive system learns the characteristics of regular traffic and notices anomalies in network usage.
- Search engines – Many different types of adaptive systems are used in search engines. Neural networks can be readily used for semantic analysis and feature extraction from texts.
- Pattern Recognition – Computer vision, speech recognition, biometrics, handwriting recognition are all areas where adaptive systems excel and completely outperform traditional methods.
- Computer Games – Neural networks are popular in game AI.