What happens if we run a support vector machine model using a kernel that does not satisfy requirements such as non-positive semi definite?
This is my flow of thought: In kernel methods $w.x$ is replaced by $\sum_i \alpha_i k(x, x_i)$. Now if k is not positive semi definite, $\exists y$ such that $k(y,y)<0$. This means there exists y such that $k(y, y) < 0$. So I feel that the classifier will become noisy, because it will keep misclassifying y. Is this correct? If so, how do I make my arguments more rigorous?
