# Improve performance of SVM

I have written some code for an SVM and the final output isn’t brilliant. The data is linearly separable so I don’t understand why the data isn’t perfectly separated. Here is the code:

from matplotlib import pyplot as plt, numpy as np

x = np.array([[-2, 4, -1], [4, 1, -1], [1, 6, -1], [2, 4, -1], [6, 2, -1]])

y = np.array([-1, -1, 1, 1, 1])

def svm_sgd(X, Y):
w = np.zeros(len(X))    # Parameters
eta = 1
epochs = 100000

# Update rules

for epoch in range(1, epochs):
for i, x in enumerate(X):
if (Y[i] * np.dot(X[i], w)) < 1:
w = w + eta * ((X[i] * Y[i]) + (-2 * (1/epoch) * w))
else:
w = w + eta * (-2 * (1/epoch) * w)
return w

w = svm_sgd(x, y)
print(w)

# Print Hyperplane —————————————

for d, sample in enumerate(x):
if d < 2:
plt.scatter(sample, sample, s=120, marker='_', linewidths=2)
else:
plt.scatter(sample, sample, s=120, marker='+', linewidths=2)

plt.scatter(2,2, s=120, marker='_', linewidths=2, color='yellow')
plt.scatter(4,3, s=120, marker='+', linewidths=2, color='blue')

x2=[w,w,-w,w]
x3=[w,w,w,-w]

x2x3 = np.array([x2,x3])
X,Y,U,V = zip(*x2x3)
ax = plt.gca()
ax.quiver(X,Y,U,V,scale=1, color='blue')
plt.show()


The bottom portion just uses matplotlib to print the data, so it’s not part of the SVM algorithm, and can therefore be ignored.

The problem ~ can anyone explain to me, based on my code, why the line doesn’t separate the data more perfectly?

• Try to replace 1/epoch with 1.0/epoch - this is important for Python 2.x, in Python 3.x it doesn't matter – MaxU Apr 16 '18 at 19:50

## 1 Answer

This is the output when I run your code. It seems to be doing fine. [  1.58876117   3.17458055  11.11863105]


Try to run it again? Perhaps you changed some parameters accidentally before posting it?

• Thank you. The weight vector I received was rounded to 1 number; yours looks much more precise. It could possibly be the fact I’m using python 2.7 or a different version of numpy that’s rounding the weight vector/not storing a significant enough value – Finn Williams Apr 16 '18 at 18:10
• Ah, i see, i was using python 3 most likely there is some subtle difference somewhere with one of the library. – Siong Thye Goh Apr 16 '18 at 18:11