X-Prop control system
From Piki
The X-Prop control system is the default control system in Synapse. First it propagates signals forward through the topology and then, in case it encounters any terminators, it doubles back, propagating a signal produced by the terminator backwards in the opposite direction.
| X-Prop | |
| Dynamic | No |
| Deployable | Yes |
Contents |
Usage
Control systems are used to coordinate signal flow between blocks and trigger updates.
Since this control system also can propagate signals backwards through the topology it is well suited for use with gradient decent based methods where the back propagation pass is used to propagate the error signal.
This control system is not dynamic and should be used with non ordered data where each sample is independent and not a function of prior or past samples as is the case for a time series. Although the X-Prop control system will delay signals of retarded links, it is not capable of matching updates of the progressively delayed back propagated error signals that appear at all back update rules prior to any memory in the forward sense (see focused learning). For such non-focused topologies to work properly, please use the dynamic X-Prop control system.
Operation
The X-Prop control system deduces the order in which to process the blocks and trigger updates. It then creates a list of sources, calculates a common signal length (by truncating excess samples) and extracts a signal with length equal to the batch length. This signal is then propagated through the topology in a forward sense with regard to the links. If any terminators are encountered, the signal flow is reversed at these blocks and a terminator signal is propagated back again through the topology. Once this cycle is completed, active update rules are triggered.
Nodes
When multiple links join at a block the signals are superpositioned (added to one another). As multiple links part from a block the signals are left untouched.
Batch length
The batch length is the number of samples that are passed through the system and used to calculate an update before that update is applied. In X-Prop it has one further implication. Too speed up calculations in few-featured data sets, multiple samples are packed together on Matrix form and run through the system as a single signal. Updating after each such signal makes the number of samples in a signal and the batch length one and same.
With very large data sets it is often preferable to use a smaller batch length then the largest possible. This will result in lower memory stress and faster training as well as faster execution. On small data sets on the other hand you will speed up training using a larger batch length then one.
Training vs. Validation
The X-Prop control system has three execution modes.
- Training - In the this mode updates are allowed and only the training channel is considered. If it contains no data, the system will not run.
- Validation - In the this mode updates are never allowed (meaning no update rules are ever triggered). The validation channel is used if it containes any data, but if it does not, the training channel is used instead.
- TrainingAndValidation - In this mode the two previous modes are interleaved. The frequency is governed by the Validation Interval setting so that every interval epoch is a training epoch. The first epoch after reseting the system is always a validation epoch.
Topology Validation
Before the control system starts propagate any signals through the topology, it performs a validation to see that everything is properly connected and that all blocks are ready to run.
If all is not well the control system will send out warnings. These warnings have a gradation to mark their severity that range from Low (Green) to Severe (red). The warnings will be displayed in the validation pane and event log.
One thing that the control system looks for is feature mismatch between the two ends of a link. If the channel policy is set to Match or Force and this occurs, the control system will try to fix it by changing the feature count of one of the blocks.
Another is whether the back propagator are compatible with the control system. If they are not, the control system can change it for a back propagator that is compatible if the Repair Policy is set to Force.
Settings
The settings can be modified using the settings browser.
| X-Prop control system settings |
|---|
|
|
GUI details
The X-Prop control system GUI show progress information and also give access to most of the settings also found in the settings browser.
Deployment
Systems are still capable of learning after they are deployed.
See also
- Control system - Article on Synapse control systems.
- List of control systems - List of control systems in Synapse.
- Dynamic X-Prop - A dynamic version of X-Prop.
- Focused learning - A way of introducing memories in static systems.