I have written some code for an SVM and the final output isn’t brilliant. The data is linearly separable so I don’t understand why the data isn’t perfectly separated. Here is the code:

from matplotlib import pyplot as plt, numpy as np

x = np.array([[-2, 4, -1], [4, 1, -1], [1, 6, -1], [2, 4, -1], [6, 2, -1]])

y = np.array([-1, -1, 1, 1, 1])

def svm_sgd(X, Y):
    w = np.zeros(len(X[0]))    # Parameters 
    eta = 1
    epochs = 100000

    # Update rules

    for epoch in range(1, epochs):
        for i, x in enumerate(X):
            if (Y[i] * np.dot(X[i], w)) < 1:
                w = w + eta * ((X[i] * Y[i]) + (-2 * (1/epoch) * w))
                w = w + eta * (-2 * (1/epoch) * w)
    return w

w = svm_sgd(x, y)

# Print Hyperplane —————————————

for d, sample in enumerate(x):
    if d < 2:
        plt.scatter(sample[0], sample[1], s=120, marker='_', linewidths=2)
        plt.scatter(sample[0], sample[1], s=120, marker='+', linewidths=2)

plt.scatter(2,2, s=120, marker='_', linewidths=2, color='yellow')
plt.scatter(4,3, s=120, marker='+', linewidths=2, color='blue')


x2x3 = np.array([x2,x3])
X,Y,U,V = zip(*x2x3)
ax = plt.gca()
ax.quiver(X,Y,U,V,scale=1, color='blue')

The bottom portion just uses matplotlib to print the data, so it’s not part of the SVM algorithm, and can therefore be ignored.

The problem ~ can anyone explain to me, based on my code, why the line doesn’t separate the data more perfectly?

  • 1
    $\begingroup$ Try to replace 1/epoch with 1.0/epoch - this is important for Python 2.x, in Python 3.x it doesn't matter $\endgroup$ – MaxU Apr 16 '18 at 19:50

This is the output when I run your code. It seems to be doing fine.

enter image description here

[  1.58876117   3.17458055  11.11863105]

Try to run it again? Perhaps you changed some parameters accidentally before posting it?

  • $\begingroup$ Thank you. The weight vector I received was rounded to 1 number; yours looks much more precise. It could possibly be the fact I’m using python 2.7 or a different version of numpy that’s rounding the weight vector/not storing a significant enough value $\endgroup$ – Finn Williams Apr 16 '18 at 18:10
  • $\begingroup$ Ah, i see, i was using python 3 most likely there is some subtle difference somewhere with one of the library. $\endgroup$ – Siong Thye Goh Apr 16 '18 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.