I'm currently trying to understand regularization for logistic regression.
So far, I'm not quite sure whether I really got it. Basically, the problem is that when we add an additional features to a model we might overfit the training set. This leads to an algorithm which perfectly matches the training set, but fails for future values (as it too exactly fits what has been there in the past).
So, this is why we introduce regularization. With this we decrease the impact of the features, so the model may produce an algorithm which does a (seemingly) worse job on the training set, but overall works better.
Is this correct?