How can I know how to interpret the output coefficients (`coefs_`) from the model sklearn.svm.LinearSVC()?

Question

I'm following Introduction to Machine Learning with Python: A Guide for Data Scientists by Andreas C. Müller and Sarah Guido, and in Chapter 2 a demonstration of applying LinearSVC() is given. The result of classifying three blobs is shown in this screenshot:

The three blobs are obviously correctly classified, as depicted by the colored output.

My question is how are we supposed to know how to interpret the model fit output in order to draw the three lines? The output parameters are given by

print(LinearSVC().fit(X,y).coef_)
[[-0.17492286  0.23139933]
 [ 0.47621448 -0.06937432]
 [-0.18914355 -0.20399596]]

print(LinearSVC().fit(X,y).intercept_)
[-1.07745571  0.13140557 -0.08604799]

And the authors walk us through how to draw the lines:

from sklearn.svm import LinearSVC  
linear_svm = LinearSVC().fit(X,y)  
...
line = np.linspace(-15, 15)  
for coef, intercept in zip(linear_svm.coef_, linear_svm.intercept_):  
    plt.plot(line, -(line * coef[0] + intercept) / coef[1])  #HOW DO WE KNOW
plt.ylim(-10, 15)  
plt.xlim(-10, 8)  
plt.show()  

The line of code with the comment is the one that converts our coefficients into a slope/intercept pair for the line:

y = -(coef_0 / coef_1) x - intercept/coef_1

where the term in front of x is the slope and -intercept/coef_1 is the intercept. In the documentation on LinearSVC, the coef_ and intercept_ are just called "attributes" but don't point to any indicator that coef_0 is the slope and coef_1 is the negative of some overall scaling.

How can I look up the interpretation of the output coefficients of this model and others similar to it in Scikit-learn without relying on examples in books and StackOverflow?

Answer 1

Here's one (admittedly hard) way.

If you really want to understand the low-level details, you can always work through the source code. For example, we can see that the LinearSVC fit method calls _fit_liblinear. That calls train_wrap in liblinear, which gets everything ready to call into the C++ function train.

So train in linear.cpp is where the heavy lifting begins. Note that the w member of the model struct in the train function gets mapped back to coef_ in Python.

Once you understand exactly what the underlying train function does, it should be clear exactly what coef_ means and why we draw the lines that way.

While this can be a little laborious, once you get used to doing things this way, you will really understand how everything works from top to bottom.

Answer 2

y = -(coef_0 / coef_1) x - intercept/coef_1 is coef_0 x + coef_1 y + intercept = 0, which is the border line separating the blob from the rest. (coef_0, coef_1) is the normal vector and the direction indicates where the blob is.