What is the difference between Perceptron and ADALINE?
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
3
The Adaline (Adaptive Linear Element) and the Perceptron are both linear classifiers when considered as individual units. They both take an input, and based on a threshold, output e.g. either a 0 or a 1.
The main difference between the two, is that a Perceptron takes that binary response (like a classification result) and computes an error used to update the weights, whereas an Adaline uses a continous response value to update the weights (so before the binarized output is produced).
The fact that the Adaline does this, allows its updates to be more repesentative of the actual error, before it is thresholded, which in turn allows a model to converge more quickly.
Have a look at this really interesting history of neural networks, which contains a small section on Adalines, along with memistors - resistors with memory, as the neurons where figuratively perceived back in the 1960's.
There are also some other answers to a similar question here.
-
$\begingroup$ Normally, delta is applied to adaline and hebb is applied to perceptron, but some sources are applied delta by adding bias to perceptron. Is there a difference in philosophy? $\endgroup$ Commented Aug 2, 2018 at 15:21
-
$\begingroup$ @Engin - The learning rules implemented by Perceptrons and Adalines have a large similarity - the weights are updated based on the (least squares) error; that essentially is equal to stochastic gradient descent. The delta in Adaline's learning method is the difference between the output and the expected output, and the hebb (named after Donald Hebb), strengthens/evolves connections to fire more often, when that appears to be a useful behaviour, and happens often. So at a philosophical/idealogical level, I believe the two methods are therefore comparable. $\endgroup$– n1k31t4Commented Aug 3, 2018 at 9:50
-
$\begingroup$ My experience with Adaline neurons with a 0/1 classfication problem ends up with the Adaline not converging. When plotting the error surface for 2 weights, I see that the surface isn't convex. This is for a binary classification dataset that has a small amount that is overlapping (so not strictly linearly separable). $\endgroup$ Commented Nov 11, 2019 at 11:53
$\begingroup$
$\endgroup$
The differences between the Perceptron and Adaline:
- The Perceptron uses the class labels to learn model coefficients.
- Adaline uses continuous predicted values (from the net input) to learn the model coefficients, which is more “powerful” since it tells us by “how much” the model is right or wrong.
$\begingroup$
$\endgroup$
First difference
In Perceptron model:
new_weight(i) = old_weight(i) + (learning_rate x target_output x input(i))
In Adaline model:
new_weight(i) = old_weight(i) + (learning_rate x (target_output - calculated_output) x input(i))
Second difference
In Perceptron model:
Activation function
y(w1 x X1 + w2 x X2... + b) =
1 if w1 x X1 + w2 x X2... + b > 0
0 if w1 x X1 + w2 x X2... + b = 0
-1 if w1 x X1 + w2 x X2... + b < 0
In Adaline model:
Activation function
y(w1 x X1 + w2 x X2... + b) =
1 if w1 x X1 + w2 x X2... + b >= Threshold
0 if w1 x X1 + w2 x X2... + b < Threshold
