Find inputs that give highest variance in output space

Question

I have a relatively simple problem that I think should have a known solution, but that I can't figure out. Any help would be greatly appreciated.

Basically, I have a function $f : \mathbb{R}^d \rightarrow \mathbb{R}^p$ which has vector input of features and vector output of images. Given a training set of combinations of the input variables and their corresponding outputs $\{x_i, y_i\}_{i=1}^N$, I would like to find the input feature (i.e. element of the inputs) that leads to the highest variability in the outputs, i.e. highest pixel-wise variance. Broadly speaking, I'm hoping to quantify the relationship between each feature and its effect on the variance of the output, to get an idea of how "sensitive" that feature is.

Methods like PCA wouldn't work because I am working in a supervised setting. Whereas PCA tries to find the directions of maximum variance of an unsupervised dataset, I am trying, in a sense, to find the directions of maximum variance in the label (output) space as it relates to the input space. Does an answer to this exist? Thanks in advance.

Answer 1

Very interesting question! I didn't get if the out put of one input vector is just one image or a set of images but this does not change the solution much.

I propose you directly learn the relation between input and the variance of output images. For simplicity, I assume you target the pixle-wise variance within an output image. Then your training data would become $\{x_i, Var(y_i)\}_{i=1}^N$ instead of $\{x_i, y_i\}_{i=1}^N$!

Then your problem can be solved as:

• Classification: You threshold the "high variance" and discritize your output into high variance and low variance classes
• Regression: Even better! you directly learn the value of variance and have more flexibility on interpreting the results at the end (e.g. the thresholding mentioned above can be applied afterwards).

I hope I understood your question right, I was clear and it could help!

Good Luck!