Madaline

Multiple Adaptive Linear Neuron

Madaline which stands for Multiple Adaptive Linear Neuron, is a network which consists of many Adalines in parallel. It will have a single output unit. Some important points about Madaline are as follows

·        It is just like a multilayer perceptron, where Adaline will act as a hidden unit between the input and the Madaline layer.

·        The weights and the bias between the input and Adaline layers, as in we see in the Adaline architecture, are adjustable.

·        The Adaline and Madaline layers have fixed weights and bias of 1.

·       Training can be done with the help of Delta rule.

Architecture

The architecture of Madaline consists of “n” neurons of the input layer, “m” neurons of the Adaline layer, and 1 neuron of the Madaline layer. The Adaline layer can be considered as the hidden layer as it is between the input layer and the output layer, i.e. the Madaline layer.

Training Algorithm

By now we know that only the weights and bias between the input and the Adaline layer are to be adjusted, and the weights and bias between the Adaline and the Madaline layer are fixed.

Step 1 − Initialize the following to start the training −

Weights

Bias

            Learning rate α

For easy calculation and simplicity, weights and bias must be set equal to 0 and the learning rate must be set equal to 1.

Step 2 − Continue step 3-8 when the stopping condition is not true.

Step 3 − Continue step 4-6 for every bipolar training pair s: t.

Step 4 − Activate each input unit as follows

Step 5 − Obtain the net input at each hidden layer, i.e. the Adaline layer with the following relation

Here ‘b’ is bias and ‘n’ is the total number of input neurons.

Step 6 − Apply the following activation function to obtain the final output at the Adaline and the Madaline layer

Output at the hidden Adaline unit

Final output of the network

Step 7 − Calculate the error and adjust the weights as follows

Case 1 − if y ≠ t and t = 1 then,

In this case, the weights would be updated on Qj where the net input is close to 0 because t = 1.

Case 2 − if y ≠ t and t = -1 then,

In this case, the weights would be updated on Q_k where the net input is positive because t = -1.

Here ‘y’ is the actual output and‘t’ is the desired/target output.

Case 3 - if y = t then

There would be no change in weights.

Step 8 − Test for the stopping condition, which will happen when there is no change in weight or the highest weight change occurred during training is smaller than the specified tolerance.