I have studied the Perceptron algorithm and now I'm trying to understand logistic regression. I know what odd ratio and Logit functions are but I don't understand why we create the following equation:
$logit(\frac{p}{1-p}) = \bar{\omega}^{T}\bar{x}$
where w is the weight vector. What does it represent, and why?
Can anyone show me, how does that work?
W is just the coefficients that we estimate to determine the contribution of each variable to the model. We attempt to estimate the log odds ratio by multiplying coefficients by the input variables that we have.
If we didn't have W, by default it would just be the 1 matrix multiplied by X, or each variable output would be assumed to affect the response equally (ignoring the scale differences between predictors).
W represents the wight matrix. This is what you try to learn, in other words to fix. X is the input data. During training, you have to provide X, feature vector, in order to learn parameters W. You learn this parameter matrix / vector, in order to have a model which represents your data. This model will be able to classify your seen and unseen data. Suppose that you have the following data. The input feature vector X is a two dimensional vector, horizontal and vertical axes represent the value for each dimension. The input features belong to either class blue or red. You try to find the separator line by learning. You learn how to specify parameter vector W to have a line which separates the data to each class. Using this procedure, after learning, you will be able to classify unseen data based on position of the data. Consider each setting of parameters will result in a different separator line. You try to find the best.
You use the inverse logit $f(x) = 1 / (1 + exp(- \omega x))$ because you want f(x) to be bound between between 0 and 1 (like a probability). If f(x) > .5 you would infer 1, 0 otherwise.
