2
$\begingroup$

The minimization problem for SVM can be written as- $$\overset{\text{min}}{\theta} C\sum_{i = 1}^{m}{[y^icost_1(\theta^Tx^i) + (1-y^i)cost_0(\theta^Tx^i)]} + \frac12\sum_{j = 1}^n{\theta_j}^2$$

Now, how can the choice of $C$ lead to underfitting or overfitting?

As I understand, parameters are chosen to make $C\sum_{i = 1}^{m}{[y^icost_1(\theta^Tx^i) + (1-y^i)cost_0(\theta^Tx^i)]}$ part $0$. And we concern ourselves with the second part.

And Andrew Ng says that a large $C$ leads to lower bias and higher variance.

How does this happen? What is the intuition behind this?

$\endgroup$
4
$\begingroup$

The C being a regularized parameter, controls how much you want to punish your model for each misclassified point for a given curve.

If you put large value to C it will try to reduce errors but at the same time it may happen that it would not perform better on test dataset hence cause overfitting.

To get to know more about effect of C in svm. Refer this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.