Oja update rule

From Piki

The Oja update rule is a Hebbian type update rule that unlike the plain Hebbian update is normalized and hence avoids instability.

Oja
Family	Hebbian
Direction	Forward
Deployable	Yes
Supervised	No

Contents 1 Usage 2 Algorithm 3 Settings 4 See also

Usage

Thanks to the the normalization, unlike the Hebbian rule, Oja's rule is stable. For Hebbian layers with one output feature, this rule can be used to produce a maximum eigenfilter. A maximum eigenfilter is a PCA transform that only keeps the main principal component.

Algorithm

Weight update:

where is the learning rate, is the input, the output and the weights.

Settings

The settings can be modified using the settings browser.

Oja update rule settings
(Learning Forward) Step: The size of the learning rate.

See also

• Update rule - General article on update rules.
• List of update rules - List of all update rules.

• Hebbian layer - A Synapse block that uses hebbian updates.
• Hebbian learning - The Hebbian update rule that is the basis for the Oja update rule.
• The talented dr Hebb part 1 - Blog tutorial on using Hebbian learning for PCA.