0
$\begingroup$

Logistic function is well known to be a good binary classificator, as it can be easily shown with this image (let x be the dot product of [x1,x2] and [w1, w2]): enter image description here

enter image description here

I am currently learning how a multiclass logistic classifier works, and this time with x1 and x2 as axes, three classes can be represented as follows (from this link):

enter image description here

According to several sites, both the "one vs the rest" and the multinomial logistic classifiers, would have one logistic regression for each of these 3 classes, with the difference of training them "together" or separately. If this is true, and we assume the next figure to be related to the previous example, I can't figure out how these 3 regressions (as in the first example) would be plotted in there.

enter image description here

As much I could figure, in the "one vs the rest" case, how to draw the logistic function for blue and yellow categories against the rest, but how would it be for the red category against the rest?

$\endgroup$
0
$\begingroup$

If you are asking how to reproduce those plots, the following technique works to visualize the decision boundaries for any model:

  1. Generate a dense grid of coordinates that fill your plotting area
  2. Score each point on your grid
  3. Plot your results, coloring observations based on your model's predictions. This serves as the background.
  4. Overlay your test data as a scatter plot

In your link, they generate the grid using np.meshgrid and constrtuct the background as a contour plot. The lines representing the decision boundaries of each respective 1-vs-all classifier is plotted using a closed form solution for logistic decision boundaries.

EDIT: To answer the question in your comment, you don't have a single $X$ dimension, and consequently your model output doesn't correspond to a curve as simple as this: it's a 3D surface. The simplest solution would be to just apply the strategy I suggested earlier (and which is also described in your link) but instead of calling .predict() to construct the background coloration, you'd call .predict_proba() and use a sequential color map (e.g. the default veridis) so color intensity corresponds to your class likelihood, giving you something which should look like this. Alternatively, you could plot surface curves. Both of these solutions are projections of the $Y$ axis, $P(Y|x1,x2)$, onto the $X_1$-$X_2$ plane. If that isn't satisfactory, you could pass scored results through PCA to combine your $X$ dimensions into a single $X$ feature -- call this $PC_1$ -- and then plot $PC_1$ vs. $P(Y|x1,x2)$. Personally I think this is significantly less informative, but it would give you an x vs. y plot.

$\endgroup$
1
  • $\begingroup$ How would we draw the logistic curves (as in the first plot, not the separation boundaries as in the second) for each of the 3 regressions of the last plot (with axes x-y, instead of x1-x2)? $\endgroup$ – freesoul Jan 4 '18 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.