I am trying to code SVM from scratch using a small toy problem that involves five support vector values. In the code below, there are 5 support vectors arbitrary chosen and denoted by the variables
s1,s2,s3,s4,s5. The support vectors are augmented with a third coordinate which is the bias = 1.
y denotes the labels for the 3 support vectors.
A is the 5 by3 design matrix that contains the values:
11 0 5 22 0 5 33 0 5 22 8 5 33 8 5
Thus the equation becomes
y = wx + b where
x is the input data. The equation for the hyperplane is
w = sum_i a_i*s_i where the
a_i's are the
alpha parameter for
Confusion: Should I take the transpose of
A in the least squares solution:
alpha = y/A' and wouldn't there be 5 values for
alpha and hence
w = [alpha1*s1+alpha2*s2+alpha3*s3 + alpha4*s4 alpha4*s5 ]? But I am getting 3 values for alpha instead of 5. Is it because there are 3 coordinates or somewhere the product is becoming zero?
% 5 support vector s1 = [1 0 1]; s2 = [2 0 1]; s3 = [3 0 1]; s4 = [2 2 1]; s5 = [3 2 1]; s_x = [1 2 3 2 3 ]; s_y = [0 0 0 2 2]; y = [-1 -1 -1 +1 +1] gscatter(s_x,s_y,y) A = [ (s1.*s1)+ (s2.*s1)+ (s3.*s1) + (s4.*s1) + (s5.*s1); (s1.*s2)+ (s2.*s2)+ (s3.*s2) + (s4.*s2) + (s5.*s2); (s1.*s3)+ (s2.*s3)+ (s3.*s3) + (s4.*s3) + (s5.*s3); (s1.*s4)+ (s2.*s4)+ (s3.*s4) + (s4.*s4) + (s5.*s4); (s1.*s5)+ (s2.*s5)+ (s3.*s5) + (s4.*s5) + (s5.*s5);] alpha = y/A' alpha1 = alpha(1) alpha2 = alpha(2) alpha3 = alpha(3) % w= sum a_i*s_i % y = wx +b w = [alpha1*s1+alpha2*s2+alpha3*s3 ]