1
$\begingroup$

So I wanted to get into the topic of 'Autoencoder', and just tested how well it would work on random vectors of size 200.

model = keras.models.Sequential()
model.add(keras.layers.Dense(50, activation='tanh'))
model.add(keras.layers.Dense(200))


model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
model.fit(vectors, vectors, epochs=50, batch_size=100)

Loss is somehow negative, and continously decreases. Accuracy is pretty much 0%, with values looking like x.xxxxxxxe-07.

The architecture is supposed to be input=200, hidden-layer=50, output=200.

It really doesn't learn anything. What did I overlook, or what is the actual right approach here?

$\endgroup$
1
  • $\begingroup$ You can't use a linear activation function at the last layer with a binary cross entropy loss function. $\endgroup$ – Shubham Panchal May 27 '19 at 16:12
2
$\begingroup$

A few things. Firstly I don't think what you are trying to do is very meaningful. An autoencoder tries to find patterns in your data and take advantage of them to be able to compress and reconstruct the data. You are trying to use it with random data, which means there will be no patterns for the autoencoder to learn. I suggest you try using MNIST or another dataset.

Another thing is that your last activation function, your loss function, your metric and possibly your data does not work together. It is hard to tell what is right since I don't know what random numbers you generated. Probably your numbers are not binary, and you would want:

model.add(keras.layers.Dense(200))


model.compile(optimizer='adam', loss='mse')

Or if your random numbers are binary then you would want:

model.add(keras.layers.Dense(200), activation='sigmoid')


model.compile(optimizer='adam', loss='binary_crossentropy')

Lastly, using accuracy as a metric does not make sense for an autoencoder. Accuracy is only useful when you are performing classification.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.