0
$\begingroup$

I applied both SVM and CNN (using Keras) on a dataset. Now, I want to compare the performance of both models.

Keras model.evaluate function predicts the output for the given input and then computes the metrics function (A metric is a function that is used to judge the performance of a model) specified in the model.compile and based on y_true and y_pred and returns the computed metric value as the output. It is obvious that SVM's accuracy calculation is different than Keras' model.evaluate.

For my dataset, Keras model (accuracy calculated based on model.evaluate) gives better accuracy than the SVM. However, a comparison between y_true and y_predict (get the y_predict by model.predict) gives a similar accuracy (this accuracy is a little lower than the accuracy return from Keras evaluate) to the SVM.

I want to know, in this scenario, how can I compare the result between both SVM and Keras model? May I conclude that Keras' model is better as its model.evaluate is giving better accuracy than SVM. What is the standard approach in this scenario to compare performance between the two models?

| improve this question | |
$\endgroup$
0
$\begingroup$

If you are training classifiers, you can check: plain accuracy percentage, F1 score and other metrics (sensitivity, specificity, etc.), you can visualise true vs predicted values with confusion matrices. There's plenty of options.

Please remember that the quality of a model can be assessed only on test data, i.e. data that you model didn't see in training phase.

| improve this answer | |
$\endgroup$
  • $\begingroup$ if I use the Keras evaluate, it gives two scores. Loss and accuracy. However, this accuracy is not exactly the same as true vs predict classes. Say, I have 10 test data, I got 8 correct predictions, 2 wrong. Its not necessarily, Keras evaluate will return accuracy as 80%. It might be a bit more as they calculate metrics on top of prediction. My concern is - in such a scenario how I will compare this accuracy to the SVM accuracy calculated from sklearn.metrics.accuracy_score? $\endgroup$ – Mohammad Nur Nobi May 2 at 0:28
  • $\begingroup$ You should use model.fit() method to generate predictions, then compare them with the predictions of the SVM model using tools such as a Confusion Matrix, and the relative metrics usually associated with it. $\endgroup$ – Leevo May 2 at 11:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.