Model the predictive relationship between images

Question

Hello fellow machine learners,

We have numerous pairs of 64 x 64 (or other dimensionality) images (maps). In each pair, the first image demonstrates a physical parameter, e.g. wind speed, at each pixel; the second shows another physical or financial parameter, e.g. temperature or insurance loss, at each pixel. We want to model the predictive relationship from the first image to the second. Our previous models typically consisted of characterizing the field using domain-specific knowledge for dimension reduction. Now my colleague wants to explore the possibility of using statistical learning methods alone.

I did a shallow research and found this could fit in a multi-output regression problem. Online forums suggested papers like Melki and Cano's "Multi-Target Support Vector Regression Via Correlation Regressor Chains", and many Python libraries. However I am a little concerned about the extreme dimensionality, i.e. 4096, and the potential failure of exploiting the spatial structure of our problem if we go down that road.

I know CNNs are suitable for image recognition tasks as such. Has anyone encountered a similar problem and attacked it with neural nets? Any suggestion would be much appreciated!

Thank you!

Answer 1

To answer my own question. A naïve approach is to train a convolutional neural net with an output layer of 4096 neurons $x_1,x_2,\ldots, x_{4096}$. Assume values in the response image are $y_1,y_2,\ldots,y_{4096}$. Let the loss function be $L=\sum_{i=1}^{4096}(x_i-y_i)^2$.