2
$\begingroup$

I have solved quite a few kaggle playground problems lately, but I can't understand how to come up with good enough model architecture which gets 0.9+ validation accuracy and without overfitting.

Is there some formula, or is it some hit and trial method for determining filters and neurons. And also I'm always using relu activation in hidden layers(read it somewhere) when will other activations be used in hidden ones.

Can you lay down some guidelines that I should follow?. Considering I mostly work with image classification problems.

And I you have some other tips as well you can share in your answer.

Improve this question
$\endgroup$
6
  • $\begingroup$ Please elaborate more on the data you are working with. What dataset is it? What is its size? What is your purpose? How are the variables? Also, what architecture did you already try? $\endgroup$
    – Leevo
    Jun 5, 2019 at 13:30
  • $\begingroup$ @Leevo I'm working simultaneously with 2 datasets namely, Fashion MNIST(87% Val_acc) and X-Ray scan Pneumonia (70% Val_acc) $\endgroup$
    – Nikhil.Nixel
    Jun 5, 2019 at 13:32
  • $\begingroup$ There is no reason to think that one model will work on both datasets, so focus on one problem at a time. $\endgroup$
    – Paul
    Jun 5, 2019 at 13:43
  • $\begingroup$ @Paul I'm aware of that and I'm using deeper model for Xray datset with (256,256)px image. Still struggling! But one more question how often should I use MaxPooling2D $\endgroup$
    – Nikhil.Nixel
    Jun 5, 2019 at 13:45
  • $\begingroup$ See my answer below: There is no generic answer, but start with something simple and try to make small changes from there: more pooling layers, fewer pooling layers, and compare the performance each time. And have a look at that Coursera course, you're going to like it. $\endgroup$
    – Paul
    Jun 5, 2019 at 13:47

1 Answer 1

Reset to default
1
$\begingroup$

There is no theoretical understanding, that would take a problem and specify the optimal network architecture for you. So no, there is no formula, I'm afraid. What does exist, are strategies to arrive at good solutions.

Good strategies are to start simple and try to make iterative improvements. Define a single KPI (error metric) that you're trying to optimize, build the simplest model you can imagine that will hopefully work a little bit, then try to improve from there. You get improvements by trying to understand why your model does not perform optimally. You need to be able to distinguish between underfitting and overfitting to make the right modifications. Study the errors that your model makes on specific images and try to make improvements to address those mistakes with changes to the model.

I think you may find this course on Coursera interesting: Structuring Machine Learning Projects.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.