0
$\begingroup$

I have created a new method to do binary image classification. I think it would be interesting to compare it to the convolutional neural network that would do the same binary classification given the same data sets. The question I have is what would be a standard architecture of such CNN model? How much should I tune it to increase the accuracy of the CNN model so it is fair to compare it to my methodology? I want this to be as fair as possible with the chance of maybe having my new methodology published. I tried googling my question but I could not stumble upon any blog about it.

I would appreciate some guidance or links.

$\endgroup$
3
$\begingroup$

If your model is simple and you don't have a lot of training data, then you need a model with few parameters to compare to, or else you won't be able to train it. Using standard CNN architectures may not be a good option in this case, because even if the point of using a CNN rather than a full-connected network is to reduce the parameters, they still have many millions of parameters. Depending on the details of the problem, I think you have to design something yourself. Have a look at this Kaggle guide on choosing a CNN architecture.

If you have a lot of input data and a complex model that you're comparing to, I would recommend using a standard architecture that people are familiar with, such as VGG-16 or VGG-19, so that a performance comparison means something to the reader.

You can get these models with pretrained weights, so you don't need to do any training necessarily. However, they are set up for multiclass classification and typically trained on ImageNet, so you need to adapt the classification part. The simplest way is to replace the last (output) layer with a single output node for your binary classification.

To make a fair comparison you then need to train the entire model on the same training dataset used for your model, and evaluate on the same test set. But I would start by retraining the final FC layers first, to get a quick preview of how it's going. Then retrain the full model for the fair comparison.

$\endgroup$
  • $\begingroup$ I have only trained my model on 30 images, so training it on VGG-X seems like an overkill. It feels especially excessive since my model is just a proof of concept. But I definitely see your point of using VGG. $\endgroup$ – Gabriele May 25 '19 at 12:12
  • $\begingroup$ I have updated the answer to be more appropriate for your scenario, but I would say that in any case, 30 training images is too few. So one way or another you would need to find a way to get more training data. If you have more images but they are unlabeled, spend a couple of days and label them. $\endgroup$ – Paul May 25 '19 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.