1
$\begingroup$

I coded a neural network from scratch in Python. I tried it with the XOR problem and it learned correctly. So I tried to encode an Autoencoder with 3 inputs (and therefore with also 3 outputs) to reduce a color (r, g, b) in one dimension. I have normalized the data from 0 to 1 so I can use activation functions like sigmoid, relu etc. I have tried many different activation functions and learning rates, but the Autoencoder error(calculated with the mean squared error) is high (the lowest I got is 0.1), although I have trained it for more than 30,000 iterations. Did I miss something? (I think so, but is it possible to reduce a color to a size with good accuracy in the first place?)

Thank you all

$\endgroup$
1
  • $\begingroup$ @MatthieuBrucher Yes, I was trying to reduce any possible color... Thank you very much! $\endgroup$ – Giuseppe Romeo Feb 13 '19 at 17:47
0
$\begingroup$

What kind of color are you trying to learn? If it's a random color, then yep, tough luck, you can't reduce the dimensionality of a 3D Euclidian manifold to less than 3D.

To be more precise, this is the case for an infinite manifold, for non-infinite one, you could get a 1D curve that goes through a 3D cube (in philosophy, it would be close to a Kohonen map, despite having just one node instead of a set of discrete nodes), but you would need a very big set of intermediate layers to make it happen.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.