How to extract crucial features to create an image

Question

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms.

My goal is to make a cat from a dog. How to do it? For now, I have a couple of ideas which I can share:

1. Use a convolutional autoencoder and train on cats, then give a picture of a dog and see the result (I suppose, it will show the most "similar" cat, so the goal is reached)

2. Use an algorithm like GAN to transfer "style". I have no idea whether it is possible or not, but looks like a working idea

Which approaches could you recommend to try out?

Answer 1

This is a bit similar, merge an image with style of another https://github.com/jcjohnson/neural-style

Maybe with the GAN you can choose cases where the discriminator predicts 50/50 to be cat or dog

Answer 2

But NN are a kind of automatic feature extractors.

So lets say you have a pre-trained net of only dogs, freeze last couple of layers and train them on cats.

BUT lets say that you really want to encode some feature, for example that dogs-cats should be brown, theres is a posibility to insert features manually. See here You can decide for yourself in which depth should it be added. To encode brown colour you could count pixels that have this spectrum in the entire picture.