1
$\begingroup$

I'm fine-tuning a InceptionResnetv2 network to get a features extractor, so I'm training a classical classifier with my data (one label/data, i'm using a softmax).

I would like to know how to choose architecture for top layers (fully connected), I read that usually Flatten -> Dense -> Dropout -> Softmax were used.

How to choose between

  • Flatten/MaxPool/AvgPool
  • Dense(256)/Dense(512)/Dense(1024)

is it purely empyric ?

$\endgroup$
1
  • $\begingroup$ In my experience the procedure is empiric, with the only exception of Global Average Pooling layers, which are usually placed deliberatly. $\endgroup$
    – Djib2011
    Sep 6, 2018 at 10:59

1 Answer 1

1
$\begingroup$

Deep learning is primarily an empirical field, best practices are found through trial and error.

Since you are exploring relatively few hyperparameter combinations, they can be compared using grid-search cross-validation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.