0
$\begingroup$

lets say i have fed an image into VGG19 pre-trained on imagenet as follows:

from tensorflow.keras.applications.vgg19 import VGG19
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.vgg19 import preprocess_input
from tensorflow.keras.models import Model
import numpy as np

base_model = VGG19(weights='imagenet')
model = Model(inputs=base_model.input, outputs=base_model.get_layer('block5_pool').output)

img_path = 'C:/shared/a5/images/training/n01518878_8432.JPEG'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

features = model.predict(x)

i then extract the values at layer5 max pool layer, which are determined by the input image, and unique for every different image that is fed into the network.

does anyone know how to reconstruct the original image based on the features extracted from a specific layer? thank you

$\endgroup$
2
$\begingroup$

One choice is to train a neural network model to take these values and output original images.

Notice that usually some data is loss in this process so it might be impossible to reconstruct the image with perfection.

You could try inverting the functional form but:

  • CNNs usually use ReLu activation which is not bijective.
  • Pooling layers throws information away (it still could be fixed if we had redundant information)
  • The weight matrix might not be inversable
$\endgroup$
2
  • $\begingroup$ thanks, are you aware of any code that does this? i know also the L-BFSG algorithm can be used but its complicated to implement. $\endgroup$ – Russell Butler Aug 6 '20 at 1:43
  • $\begingroup$ If you check CapsNet original paper, they do it using a simple fully connected NN $\endgroup$ – Pedro Henrique Monforte Aug 6 '20 at 1:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.