3
$\begingroup$

I tried to standardize the training data with samples of 629,145 rows and 24 features:

from sklearn import datasets
import pandas as pd
from sklearn import svm
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import StandardScaler

df = pd.read_csv('mydata.csv', dtype='object')

#manually choosing 24 features
X=df.loc[:, ['Bwd Pkt Len Min','Subflow Fwd Byts','TotLen Fwd Pkts','TotLen Fwd Pkts','Bwd Pkt Len Std','Flow IAT Min',
             'Fwd IAT Min','Flow IAT Mean','Flow Duration','Flow IAT Std','Active Min','Active Mean','Fwd IAT Min',
             'Bwd IAT Mean','Fwd IAT Mean','Init Fwd Win Byts','ACK Flag Cnt','Fwd PSH Flags','SYN Flag Cnt','Fwd Pkts/s',
             'Bwd Pkts/s','Init Bwd Win Byts','PSH Flag Cnt','Pkt Size Avg']]
Y= df['Label'] 

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.4,random_state=42) # 60% training and 40% test
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

#Create a svm Classifier
clf = svm.SVC(kernel='rbf') # not linear Kernel
clf.fit(X_train, y_train)

Already 6 hours and SVM does not converge. Same data is converged with other algorithms very fast as RF and I know it is quite normal as SVM considered computationally high algorithm compared to KNN and RF.. I read quite a bit of questions/answers and articles.

  1. I wonder how can I track and analyze the problem visually (probably plotting some graphs as cost function or ?).
  2. Can parameter tuning (C parameter) be of the help and speed up this?
  3. What could be your advice?

Thanks very much

$\endgroup$
  • $\begingroup$ Have you evaluated the performance of your scaled vs. non-scaled on the test set? Also, you should tune the regularization parameter using a validation set. Lastly, have you tried logistic regression? $\endgroup$ – Wes Feb 18 at 0:28
  • 1
    $\begingroup$ My advice would be to first subsample your data... Do exactly the same on 10% of the data or even just 1000 rows, to see if it converges. If not, you may have a subtle bug (I cannot spot one at first glance). $\endgroup$ – n1k31t4 Feb 18 at 23:33
  • $\begingroup$ indeed, I am exactly doing this. 300,000 samples converge successfully $\endgroup$ – joy lee Feb 19 at 17:11
1
$\begingroup$

I do not program on python. Nevertheless, I would say the key relies in the number of samples (629,145). They are many. The SVM has to test them to pick which are good support vectors for data partition/regression and given the size of the dataset there are a lot of alternatives. That issue plus the number of different C, gamma/sigma and, perhaps, epsilon (not sure you are classifying or regressing) tested during the optimisation of the SVM hinders convergence. There are people who used clusters instead of the original dataset to train the SVM. For instance: Barros de Almeida, M., de Padua Braga, A., Braga, J.P., 2000. SVM-KM: speeding SVMs learning with a priori cluster selection and k-means, in: Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks. Rio de Janeiro, (Brazil), pp. 162–167. can be a good starting reference but, I am aware there a others. There is at least one R package based on that idea (LinearizedSVR). In addition, there are analytical methods to infer the values of the parameters C, gamma/sigma and epsilon (for regression) based on the characteristics of the training dataset. That means no optimisation is necessary, although there is a friend that still tunes gamma/sigma but fixes C fallowing these approaches. I think there is available code (at least for R) somewhere in the net. The refs are: Cherkassky, V., Ma, Y., 2004. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Networks 17 (1), 113–126. 10.1016/S0893-6080(03)00169-2 for regression and, Keerthi, S.S., Lin, C.-J., 2003. Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel. Neural Comput. 15 (7), 1667–1689. 10.1162/089976603321891855 for classification SVMs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.