I am using CNN autoencoder to create a state representation layer which I will later be feed into my Reinforcement Agent. So I trained my CNN autoencoder and it is giving nice state representations. But I have following questions,

  1. Can my autoencoder layer be overfitted
  2. If there's a overfit will it cause rubish information in my bottleneck layer?

Yes to both of your questions. Your autoencoder can overfit and this will cause your bottleneck to store useless information (besides any useful information it already stores).

Some ways to prevent this is:

  • Find a larger dataset, or augment the current.
  • Add noise to the input (see de-noising autoencoders).
  • Regularization (e.g. early stopping, sparsity constraints)
  • $\begingroup$ is there any way of checking? My Auto -Encoder can only use state represented by an image in the simulator. Which is just an indoor scenario? So there's not much of a pattern between states. It it's kitchen it has stuff similar to the kitchen. Training with auxiliary losses can help to get rid of overfitting or not? $\endgroup$ Sep 20 '18 at 4:35
  • $\begingroup$ Also what do you mean by "beside any useful information it already stores" this means somehow to reconstruct bottleneck will keep useful information right ? even though overfit $\endgroup$ Sep 20 '18 at 6:41
  • 2
    $\begingroup$ @ShamaneSiriwardhana Yes, if you have labelled data, you can have an auxiliary output with an auxiliary loss and that will help the autoencoder identify which elements from the input image are necessary for classification. By "useful information" I mean any useful relations the autoencoder has managed to extract and are stored in the bottleneck layer. Besides those the autoencoder might also model the "noise" in the training set, which is what will cause it to overfit $\endgroup$
    – cat91
    Sep 24 '18 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.