1
$\begingroup$

I'm following Introduction to Machine Learning with Python: A Guide for Data Scientists by Andreas C. Müller and Sarah Guido, and in Chapter 2 a demonstration of applying LinearSVC() is given. The result of classifying three blobs is shown in this screenshot:

enter image description here

The three blobs are obviously correctly classified, as depicted by the colored output.

My question is how are we supposed to know how to interpret the model fit output in order to draw the three lines? The output parameters are given by

print(LinearSVC().fit(X,y).coef_)
[[-0.17492286  0.23139933]
 [ 0.47621448 -0.06937432]
 [-0.18914355 -0.20399596]]

print(LinearSVC().fit(X,y).intercept_)
[-1.07745571  0.13140557 -0.08604799]

And the authors walk us through how to draw the lines:

from sklearn.svm import LinearSVC  
linear_svm = LinearSVC().fit(X,y)  
...
line = np.linspace(-15, 15)  
for coef, intercept in zip(linear_svm.coef_, linear_svm.intercept_):  
    plt.plot(line, -(line * coef[0] + intercept) / coef[1])  #HOW DO WE KNOW
plt.ylim(-10, 15)  
plt.xlim(-10, 8)  
plt.show()  

The line of code with the comment is the one that converts our coefficients into a slope/intercept pair for the line:

y = -(coef_0 / coef_1) x - intercept/coef_1

where the term in front of x is the slope and -intercept/coef_1 is the intercept. In the documentation on LinearSVC, the coef_ and intercept_ are just called "attributes" but don't point to any indicator that coef_0 is the slope and coef_1 is the negative of some overall scaling.

How can I look up the interpretation of the output coefficients of this model and others similar to it in Scikit-learn without relying on examples in books and StackOverflow?

$\endgroup$
  • $\begingroup$ I think the only hint is that is says, the coefficients belong to the primal problem which is briefly stated in the user guide under 1.4.7.1. It then follows probably more or less from the definition of the separating hyperplane. If your question is whether there are some more details about the coef_ attributes, then the answer is probably no. The documentation, in general, is not very detailed regarding implementation details. $\endgroup$ – oW_ Mar 29 '17 at 23:57
2
$\begingroup$

Here's one (admittedly hard) way.

If you really want to understand the low-level details, you can always work through the source code. For example, we can see that the LinearSVC fit method calls _fit_liblinear. That calls train_wrap in liblinear, which gets everything ready to call into the C++ function train.

So train in linear.cpp is where the heavy lifting begins. Note that the w member of the model struct in the train function gets mapped back to coef_ in Python.

Once you understand exactly what the underlying train function does, it should be clear exactly what coef_ means and why we draw the lines that way.

While this can be a little laborious, once you get used to doing things this way, you will really understand how everything works from top to bottom.

| improve this answer | |
$\endgroup$
1
$\begingroup$

y = -(coef_0 / coef_1) x - intercept/coef_1 is coef_0 x + coef_1 y + intercept = 0, which is the border line separating the blob from the rest. (coef_0, coef_1) is the normal vector and the direction indicates where the blob is.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yup, by seeing that equation we can figure out what the coefficients mean. My question is how could we have known that without looking at the example in the Müller/Guido book and back it out? $\endgroup$ – user1717828 Apr 3 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.