# Neural network, Support Vector Machine or something else to classify into 7 groups

I'm an experienced developer but I'm only starting to discover data science. I have a data set consisting of 62 parameters for each row and each row in that data set belongs to one of 7 groups (numbered from 0 to 6).

x0  x1  x2  x3  x4  x5  x6  x7  x8  x9  x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x20 x21 x22 x23 x24 x25 x26 x27 x28 x29 x30 x31 x32 x33 x34 x35 x36 x37 x38 x39 x40 x41 x42 x43 x44 x45 x46 x47 x48 x49 x50 x51 x52 x53 x54 x55 x56 x57 x58 x59 x60 x61 y
b4d8a653ea  16a14a2d17  06330986ed  ca63304de0  a62168d626  1746600cb0  1   1   -0.6887062641683063 7e5c97705a  e5df3eff9b  91bb549494  e33c63cf35  3694.0  6e40247e69  617a4ad3f9  718c61545b  c26d08129a  634e3cf3ac  dd9c9e0da2  17c99905b6  513a3e3f36  9aba4d7f51  40.57961189718329   -0.11269265451935975    -0.17219069579806134    1.1666666666666663  1.6745384722167482  0.6308894281294708  37.0    1.294921875 55.0    0.16666666666666666 10.0    0.0 0.0 1.0 9.0 0.0 1.0 23.0    Бер.67  0.12    1.935   02.Лют  0.625   0.25    0.125   0.0 0.813   0.07400000000000001 0.634   0.5479999999999999  0.2353332208066929  0.2649521447821752  0.0 0.3333333333333333  0.3333333333333333  0.3333333333333333  0.0 0.0 9.0 2
467f9617a3  16a14a2d17  06330986ed  ca63304de0  b7584c2d52  1746600cb0  1   1   0.8708708626728477  5624b8f759  fa0b797a92  669ea3d319  f178803074  18156.0 01ede04b4b  617a4ad3f9  718c61545b  d342e2765f  bb20e1ca06  8a6c8cef83  1b02793146  992153ed65  9aba4d7f51  28.76550293196428   2.6122849082704658  2.1590908057403015  4.0 1.7107137612171608  1.7135384162978815  0.16666666666666666 0.027669270833333325    109.0   0.0 31.0    0.0 0.0 1.0 244.0   1.0 1.0 68.0    17.25   0.57    3.452   4.0 0.409   0.619   0.579   0.248   0.34600000000000003 0.541   0.522   0.0 1.782346041542782   1.3224094711633876  0.011647254575707157    0.39767054908485855 0.2396006655574044  0.2495840266222961  0.06821963394342763 0.033277870216306155    601.0   4
190436e528  16a14a2d17  06330986ed  ca63304de0  b7584c2d52  1746600cb0  1   1   0.4376549941058605  5624b8f759  152af2cb2f  91bb549494  e33c63cf35  1178.0  cc69cbe29a  617a4ad3f9  e8a040423a  c82c3dbd33  ee3501282b  199ce7c484  5f17dedd5c  5c5025bd0a  9aba4d7f51  24.94393348850157   -0.8146595838365664 -0.7083080633874904 01.Тра  -0.5124221809900756 -0.7339666422629345 0.3333333333333333  14.837727864583336  11.0    0.0 24.0    0.0 0.0 1.0 29.0    0.0 3.0 11.0    Кві.42  0.15    0.161   0.2 1.0 1.0 1.0 1.0 1.0 0.52    0.5329999999999999  0.835   -0.5865396521883026 0.6724356815192951  0.0 0.6060606060606061  0.12121212121212124 0.21212121212121213 0.060606060606060615    0.0 33.0    3


I don't know the relationship between parameters or what influence they have on the group number (y). I need to create and train a model that will predict a group based on 62 variables, with a high success rate (> 80%). Where should I start?

Welcome to the site, @intellion! I can recommend a couple of things for you. First, if you think you'll continue your interest in data science and will be working on problems like this in the future, it is worthwhile to start familiarizing yourself with the field. There are a variety of introductory textbooks and courses you can choose from. If you have a strong mathematical background, I'd recommend Tibshirani's "The Elements of Statistical Learning", which is a solid classic in the field. If you're approaching this from a more application-oriented angle, I'd recommend something written specifically for the language you'd like to work in. If you plan on using R, there's "Machine Learning with R"; if you plan on using python, there's "Machine Learning in Python." There are of course a variety of free online resources as well.

To your question, you could certainly solve your problem with Support Vector Machines (SVMs) or Neural Networks, but I don't think that's the first place to start. To me, this looks like a classic multiple regression application situation (for an overview for doing this in R, see this or this textbook). If you really want to try something in machine learning, perhaps decision trees are a good place to start, as they'll allow you to easily assess the influence of variables on the final model fit (e.g., using Gini Impurity). I've found the rpart library in R to be quite nice to work with, but it can be a little slow for larger data. You could also use sklearn in python, if that's your preferred language.

• Thank you for such a warm greeting! I'm planning to continue studying Data Science - it is such a fascinating subject and with all the sensors and Internet of Things the demand for DS specialists will only increase :) I'm pretty good in math so I will definitely read The Elements of Statistical Learning. As for the application aspect of it - I'm know Python already(I know there are many good statistical and machine learning libraries for it) and heard R is relatively easy to pick up. Which one would you recommend? – intellion Oct 18 '15 at 9:26
• rpart is chewing my data for almost 50 hours now, on a quad-core i7 :) Are there ways to speed it up? – intellion Oct 20 '15 at 9:44
• @intellion: That's great! I think Python and R are both great for data science. I tend to use Python on text-mining types of problems, and use R for more traditional regression and statistics, but that's just my personal preference. – Kyle. Oct 20 '15 at 16:28
• @intellion: Yes, I've encountered that as well. I've found that to be fairly normal, but I've gotten some performance boost from using rpart.control calls. E.g., rpart(Y ~ X, control=rpart.control(maxcompete=1, maxsurrogate=0). Make sure and check the documentation so you don't use a set of parameters that will break your particular application of rpart! – Kyle. Oct 20 '15 at 16:30

It looks like your data includes strings and values. I suggest that you start with something simple as Logistic Regression. The strings can be hashed and the real values can be fed in as is.

Split your dataset randomly in say 70% training and the remaining 30% for testing. Fit a model on the training data and evaluate the performance on the test data. This will give you a rough estimate of your classifier.

The Vowpal Wabbit software package has everything you need to do the classification (and more). The format is simple, the learning is lightning fast - everything you need to experiment.

• Thank you for an excellent suggestion about randomly splitting training data. That's what I'm doing now. Vowpal Wabbit seems to be really popular and is often used during competitions on Kaggle. Will give it a try, thanks! – intellion Oct 18 '15 at 9:36
• I searched over online docs for VW and couldn't find a good example for decision trees and/or random forests. Could you give an example? – intellion Oct 18 '15 at 12:33
• VW does not implement decision trees or random forests. You can start with the simple logistic regression. If you notice that your Logistic Regression overfits on your data, then you can use ensembles or random forests to counter this effect. Always start with simple things. – Vladislavs Dovgalecs Oct 18 '15 at 19:20
• Thanks. Logistic Regression requires labels {-1,1} but my data has 7 classes represented with integers from 0 to 6. How should I deal with that? – intellion Oct 19 '15 at 13:23
• @intellion Logistic Regression is not limited to binary classification. If you are still using VW, take a look at --oaa option. As an argument you specify the number of classes. You can also have string labels for your classes but then you must specify to VW the exhaustive list of labels. – Vladislavs Dovgalecs Oct 19 '15 at 16:20