I am trying to reconstruct an image from a dense layer with is a concatenation of outputs from a 1) convolutional network with image inputs; and 2) dense layer with numerical inputs
The concatenated 1D tensor is fed to a dense layer which I need to reconstruct as an image.
The code I am using right now is as so:
merge_output = tf.keras.layers.concatenate([convolutional_model_output, numerical_model_output])
densem1 = Dense(8092, activation='relu')(merge_output)
#densem2 = Dense(512, activation='relu')(densem1)
densem2 = Dense(self.image_width*self.image_height*3, activation='relu')(densem1)
reshapem1 = Reshape(target_shape=(self.image_height, self.image_width, 3))(densem2)
convm1 = Conv2D(filters=32, kernel_size=3, padding="same" , activation='relu')(reshapem1)
convm2 = Conv2D(filters=3, kernel_size=3, padding="same" , activation='relu')(convm1)
However, the training is not able to converge to the true output and looks like a kaleidoscope of weird colors.
Is there something wrong with my approach? I do realize that concatenating convolutional and numerical features and reshaping them as an image might be a possible pain point, is there a better way to represent the layer or the specific problem?
Thanks in advance.