In machine learning, the perceptron is an algorithm for supervised learning of binary classifiers. A binary classifier is a function which can decide whether or not an input, represented by a vector of numbers, belongs to some specific class.

on wiki page, the problems section includes regression without more explanation.

enter image description here

can a perceptron used for regression?


The deep feedforward neural networks used for regression are nothing but multilayer perceptron architectures.


Originally, perceptrons were used as binary classifiers i.e to classify binary labels ( 0 or 1 ). But, if no non-linear activation function is applied to the dot product of the features and weights, then it is simply a linear regressor.

If the linear function is $f(x) = x$ and $N$ is the number of features then,

$\Large y = \sum_{i=0}^N x_i w_i$

or using vector notation,

$\Large y = \vec{x}.\vec{w} ...(1)$

For multiple regression, we use the below equation with mean-squared error loss function to optimize the $\beta$ parameters,

$\Large y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 ... \beta_n X_n$

Let ,

$\Large \vec{W} = [ \beta_1 \ \beta_2 \ \beta_3 \ ... \ \beta_n ] \\ \vec{X} = [ X_1 \ X_2 \ X_3 \ ... \ X_n ] $

Where $N$ is the number of features, $\vec{X}$ is the feature vector and $\vec{W}$ is the weights vector. Then,

$\Large y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 ... \beta_n X_n = \beta_0 + \vec{W}.\vec{X} ...(2) $

Where, $\beta_0$ is also called as the bias coefficient in Artificial Neural Networks. You can relate equations 1 and 2 and understand the concept.

Hence, $\Large y = \vec{x}.\vec{w} + bias $ generally represents a hyperplane which is used in linear regression.


Yes a perceptron (one fully connected unit) can be used for regression. It will just be a linear regressor. If you use no activation function you get a regressor and if you put a sigmoid activation you get a classifier.

Actually, with neural networks, classification is a special case of regression where we "regress" the probability of belonging to a given class. That's why the loss function for classification is called "logistic regression".


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.