how to reconstruct image from feature space of convolutional neural network?

Question

lets say i have fed an image into VGG19 pre-trained on imagenet as follows:

from tensorflow.keras.applications.vgg19 import VGG19
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.vgg19 import preprocess_input
from tensorflow.keras.models import Model
import numpy as np

base_model = VGG19(weights='imagenet')
model = Model(inputs=base_model.input, outputs=base_model.get_layer('block5_pool').output)

img_path = 'C:/shared/a5/images/training/n01518878_8432.JPEG'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

features = model.predict(x)

i then extract the values at layer5 max pool layer, which are determined by the input image, and unique for every different image that is fed into the network.

does anyone know how to reconstruct the original image based on the features extracted from a specific layer? thank you

Answer 1

One choice is to train a neural network model to take these values and output original images.

Notice that usually some data is loss in this process so it might be impossible to reconstruct the image with perfection.

You could try inverting the functional form but:

• CNNs usually use ReLu activation which is not bijective.
• Pooling layers throws information away (it still could be fixed if we had redundant information)
• The weight matrix might not be inversable