Competitive learning
From Piki
Competitive learning is a class of unsupervised learning algorithms based on the idea of adjusting a weight matrix in such a way that the weights represent cluster centers.
Contents |
Basic concepts
The basic unit of operation in competitive learning is most commonly known as an "artificial neuron" which is a very misleading name as it has nothing at all in common with either biological neurons or the artificial neurons found in regular artificial neural networks. A more suitable name would perhaps be "artificial particle" or "agent", but in this text we will use another name that is also sometimes used in the literature: "unit". While it may be a very non-descriptive term, it is better than the misleading "artificial neuron" and more convenient than making up new names for it.
These "units" are essentially special type of data points. The big difference between them and the regular data points is that the units are dynamic – they have active behavior and change their position based on rules and algorithms. The most simple type of unit, the basic competitive unit can be seen as a 'magnetic' data point – gravitating towards other data points. It is attracted to data, and the nearer another data point lies, the more attracted the unit gets to it and moves towards it. If a unit is close to a cluster of data points, it will gravitate towards it. Other, more distant clusters will attract it as well, but with a much weaker force.
For the mathematically inclined, the change in the position of the unit for each data point can be expressed as follows:
The vector 
is the position of the unit and the vector x is the position of the data point. The 
is a factor called “learning rate" and it regulates how fast the unit will move towards the data point. The function 
is a distance scaling function – the larger the distance between p and x, the smaller d(p,x) will be. Expressed in plain English, it would be something like this: the change in position of the unit equals a learning rate multiplied with the difference between the positions of the unit and the data point scaled by the distance between the unit and the data point.
Observe the animated gif below: the small black dots are data points while the large green dots are the units. Look how they move:
| Competitive learning illustrated |
|---|
|
The time honored tradition in the field of neural networks is that for each slight modification of an existing algorithm, a brand new name is given to the algorithm. Competitive learning is no exception – there are tons of minor varieties that go under different names, but they are all essentially based on the same basic principle of units moving in the general direction of nearby data points.
Algorithm
where is the learning rate, 
is the input vector and 
is the weight vector.
Competitive update rules in Synapse
Synapse includes a numeber of different competitive algorithms including:
- Competitive update rule -Basic form of competitve learning
- Neural gas update rule -A more advanced stochastic algorithm
- Kohonen update rule -For use with Self-Organizing Maps
See also
- Unsupervised learning
- Competitive layer - Synapse block that can use various competitive update rules.