0
$\begingroup$

I am trying to code SVM from scratch using a small toy problem that involves five support vector values. In the code below, there are 5 support vectors arbitrary chosen and denoted by the variables s1,s2,s3,s4,s5. The support vectors are augmented with a third coordinate which is the bias = 1. y denotes the labels for the 3 support vectors. A is the 5 by3 design matrix that contains the values:

11  0   5
22  0   5
33  0   5
22  8   5
33  8   5

Thus the equation becomes y = wx + b where x is the input data. The equation for the hyperplane is w = sum_i a_i*s_i where the a_i's are the alpha parameter for i=1,2,3,4,5.

Confusion: Should I take the transpose of A in the least squares solution: alpha = y/A' and wouldn't there be 5 values for alpha and hence w = [alpha1*s1+alpha2*s2+alpha3*s3 + alpha4*s4 alpha4*s5 ]? But I am getting 3 values for alpha instead of 5. Is it because there are 3 coordinates or somewhere the product is becoming zero?

% 5 support vector
s1 = [1 0 1];
s2 = [2 0 1];
s3 = [3 0 1];
s4 = [2 2 1];
s5 = [3 2 1];

s_x = [1 2 3 2 3 ];
s_y = [0 0 0 2 2];

y = [-1 -1 -1 +1 +1]

gscatter(s_x,s_y,y)


A = [ (s1.*s1)+ (s2.*s1)+ (s3.*s1) + (s4.*s1) + (s5.*s1);
      (s1.*s2)+ (s2.*s2)+ (s3.*s2) + (s4.*s2) + (s5.*s2);
      (s1.*s3)+ (s2.*s3)+ (s3.*s3) + (s4.*s3) + (s5.*s3);
      (s1.*s4)+ (s2.*s4)+ (s3.*s4) + (s4.*s4) + (s5.*s4);
      (s1.*s5)+ (s2.*s5)+ (s3.*s5) + (s4.*s5) + (s5.*s5);]
  
  alpha = y/A'
  alpha1 = alpha(1)
  alpha2 = alpha(2)
  alpha3 = alpha(3)
 
  
  % w= sum a_i*s_i
  % y = wx +b
  w = [alpha1*s1+alpha2*s2+alpha3*s3 ]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.