# Autoencoder not learning walk forward image transformation

I have a series of 15 frames with (60 rows x 50 columns). Over the course of those 15 frames, the moon moves from the top left to the bottom right.

I am attempting a walk forward autoencoder where:

• The input data is a 60x50 image.
• The evaluation label is a 60x50 image from 2 frames later.
• All data is scaled between 0-1.
    model = keras.models.Sequential()

# last layer tried sigmoid with BCE loss.
# last layer tried relu with MAE.


Tutorials say to use a final layer of sigmoid and BCE loss, but the values I'm producing must not be between 0-1 because the loss goes way negative.

If I use a final layer of relu with MAE loss it claims to learn something.

But the predicted image is notttt great:

• Am I just not using enough layers for that amount of pixels? Jul 31 at 12:20
• Please explain the experiment. How are you generating a new image? Is the image count=15? Aug 8 at 10:02
• Yes, the count is 15. I screenshotted a video of the moon and cropped them to the exact dimensions using PIL. Aug 8 at 10:47
• Here is exactly how I am running the experiment aiqc.readthedocs.io/en/latest/notebooks/… Aug 8 at 16:59

Anyways, as you keep ~95% untouched in your transformation, a ResNet would probably be very helpful! If you use an hourglass network, you shall see it quickly converging into a self-encoder (which is ~95% right) and than focus all the training just on the difference (the moon part).