Say I have an algorithm that accepts as input structured parameters of the following format, generates an audio clip and then a 512x512 spectrogram out of it:
[ param1 = numeric_value, param2 = numeric_value, ..., param100 = numeric_value ]
How can I do the opposite? That is, provide a 512x512 spectrogram and get a set of random candidate parameter values that would yield a similar but random spectrogram if fed into the algorithm?
In terms of text-to-image models, I see this as the opposite problem. Instead of using a prompt to generate a random matching image, I would like to use an image to obtain a random matching "prompt" that's not natural language (i.e. structured and numeric).
Regarding the algorithm, we can assume that the amount of changes in a resulting spectrogram is proportional to the amount of changes in the parameter values. That is, close to no parameter changes will yield a very similar spectrogram than the previous, making training somewhat possible. The algorithm is also deterministic and will always produce the same output for a given input.
Is this possible? GANs seemed to be a nice architecture for this knowing that I can generate as many "real" training data as I want using the algorithm. The generator would generate a random list of structured parameters from a spectrogram, whereas the discriminator would check whether the parameter list is real or fake (i.e. coming from my training set or the generator). In practice though, I'm not sure how I would implement any of this knowing that GANs are usually not used that way (they usually produce images, not the other way around). There might also be a better architecture for this use case that I'm not aware of (e.g. latent space encoder).