# Visualizing Support Vector Machines (SVM) with Multiple Explanatory Variables

I was wondering if anyone was aware of any methods for visualizing an SVM model where there are more than three continuous explanatory variables. In my particular situation, my response variable is binomial, with 6 continuous explanatory variables (predictors), one categorical explanatory variable (predictor). I have already reduced the number of predictors and I am primarily using R for my analysis.

(I am unaware if such a task is possible/ worth pursuing.)

Does it matter that the model is created in the form of SVM?

If no, I have seen a clever 6-D visualization. Its varieties are becoming popular in medical presentations. 3 dimensions are shown as usual, in orthographic projection. Dimension 4 is color (0..255) Dimension 5 is thickness of the symbol Dimension 6 requires animation. It is a frequency of vibration of a dot on the screen. In static, printed versions, one can replace frequency of vibration by blur around the point, for a comparable visual perception.

If yes, and you specifically need to draw separating hyperplanes, and make them look like lines\planes, the previous trick will not produce good results. Multiple 3-D images are better.

• This was for a homework assignment related to SVM, I didn't need to create a Visualization, I just was curious if it had ever been dome with anything outside 3 dimensions. I am not entirely sure I know what you are referring to when you say multiple 3-D images, just so I am clear, I take it to mean creating several models with only 3 explanatory variables in them, correct? – k6adams Nov 18 '14 at 20:17
• Correct. I sometimes also show images with individual variables on two axis, and a weighted average of the remaining variables (in your case, 4 of them) on the third axis. This can show dense clusters of data that were not obvious by examination of separate 3-D pictures. – Oleg Roderick Nov 19 '14 at 0:23
• I'm downvoting this answer because it's far too specific, to the point where I think it's incomplete. Rather than this particular 6D visualization (which, I'm sorry to say, is not new at all), you can just apply the general principle of using color/shape/etc. to represent higher dimensions. It's also important to realize that you can in fact have too many dimensions on one plot, and that mapping features to non-spatial scales needs case-by-case judgement. – shadowtalker Nov 21 '14 at 18:58

Have you looked into tourr package in R. This package does hyperplane reduction. In addition it has an optimizer that tries to find the best reduction.

There is a very nice video in https://www.youtube.com/watch?v=iSXNfZESR5I That shows what R is capable even beyound tourr package.

• I will have to try this out, thanks for the tip! I would vote both you and Oleg up but I don't have enough reputation points. – k6adams Nov 20 '14 at 21:03

Dimension reduction (like PCA) is an excellent way to visualize the results of classification on a high-dimensional feature space.

The simplest approach is to project the features to some low-d (usually 2-d) space and plot them. Then either project the decision boundary onto the space and plot it as well, or simply color/label the points according to their predicted class. You can even use, say, shape to represent ground-truth class, and color to represent predicted class.

This is true for any categorical classifier, but here's an SVM-specific example: http://www.ece.umn.edu/users/cherkass/predictive_learning/Resources/Visualization%20and%20Interpretation%20of%20SVM%20Classifiers.pdf

In particular, see figures 1a and 2a.