I have a multilabel data which I've trained using different classifiers with MEKA (multilabel version of WEKA) and the evaluation results (e.g. accuracy) that MEKA gives me are different from those I get from the same classifiers of scikit-learn.
I am using Binary Relevance method (which is both in MEKA and scikit-multilearn), because of my data being multilabel. And for instance, I am using SVM which is SMO in MEKA (and WEKA) and linearSVC in sklearn.
Now the accuracy I get from 10-fold cross-validation with BR>SMO in MEKA is about 50% where I get something around 20% accuracy from BinaryRelevanvce>LinearSVC. Other evaluation measures (e.g. f1-score, hamming loss etc.) don't match as well.
I tried to give all the parameters I could find in the input of LinearSVC, the same value as those of SMO but there's no luck. My Python code would be like this:
from skmultilearn.problem_transform import BinaryRelevance from sklearn.svm import LinearSVC my_classifier = BinaryRelevance( classifier=LinearSVC(C=1, random_state=1, tol=0.001, max_iter=10000, dual=False), require_dense=[True, True])
I must add that using BR>SMO in MEKA to train and evaluate the model would take almost an hour on my data, but sklearn's linearSVC (with skmultilern's BinaryRelevance) takes less than 10 minutes on the same data (hence the lower accuracy).
And by the way, I'm using
accuracy_score method in
sklearn.metrics for accuracy.
So my question is what am I missing? How can I get the same accuracy from SVM in MEKA and scikit-learn?