2
$\begingroup$

I have about 500k rows of data. I'm new to data science and I'm trying to train a model utilizing support vector machines as part of my analysis. On my little macbook pro, it seems endless. Right now I'm using an RBF kernel and am not seeing an end to this computation.

I do however have access to 100+ compute nodes with 16 cores a piece. My problem is, I don't know how to utilize this to my extent as I am lacking knowledge on how I should approach this SVM. Right now I am using scipy.

SciPy Code

def makeModelandPrediction(trainData, trainLabel, testData):
    model = svm.SVC(
        C=1.0,
        cache_size=200,
        class_weight=None,
        coef0=0.0,
        decision_function_shape=None,
        degree=3,
        gamma='auto',
        kernel='rbf',
        max_iter=-1,
        probability=False,
        random_state=None,
        shrinking=True,
        tol=0.001,
        verbose=False,
        )
    model.fit(trainData, trainLabel)
    prediction = model.predict(testData)
    return prediction

I've already preprocessed the data and done a 70/30 train/test split on the data. Can someone point a beginner in the right direction?

$\endgroup$
  • 3
    $\begingroup$ Use the SGD classifier with kernel approximation; it's the only way to scale SVM. Welcome to DataScience.SE! $\endgroup$ – Emre Nov 23 '16 at 0:53
  • $\begingroup$ How long have you let it run for? A little testing with subsets of your data might give you an idea whether this is going to take hours (bearable?), days (barely bearable) or weeks (unbearable) on your full data. $\endgroup$ – Spacedman Nov 23 '16 at 17:08
1
$\begingroup$

If you want to stick to SVC(sklearn) , you can use Ensembling we have many ways of ensembling good guide is here : http://mlwave.com/kaggle-ensembling-guide/ for your problem you can train multiple SVC models on different subset of your data,and as quick solution you can combine them whit BaggingClassifier like this :

clf = BaggingClassifier(SVC(C=1.0,
        cache_size=200,
        class_weight=None,
        coef0=0.0,
        decision_function_shape=None,
        degree=3,
        gamma='auto',
        kernel='rbf',
        max_iter=-1,
        probability=False,
        random_state=None,
        shrinking=True,
        tol=0.001,
        verbose=False,
        ))

If you want to use each record is used only once for training in the BaggingClassifier, set the bootstrap parameter to False. now you can parallel each of this models in on core of your cluster.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.