# Encoding correlation

I have rather theory-based question as I'm not that experienced in encoders, embeddings etc. Scientifically I'm mostly oriented around novel evolutionary model-based methods.

Let's assume we have data set with highly correlated attributes. Usually encoders are trained to learn representation in lesser number of dimensions. What I'm wondering about is quite the opposite. Would it be possible to learn encoding to higher number of dimensions but less correlated (wishfully non-correlated)? The idea is to turn less-dimensional, very tough problem to high-dimensional but easier one. Kinda unwrap those intricate correlations using NN and decode solutions later.

Edit 1 Of course we assume we know correlation mapping really good. How exactly could I use correlation mapping to unwrap it? Is it fundamentally possible to unmap attribute dependencies?

• like the kernel trick?
– oW_
Jan 13 '20 at 17:16
• Considering vanilla ML kernel makes classes linearly separable. I would like feature attributes to be less correlated. I'll investigate it, haven't thought about it, thanks! Jan 13 '20 at 17:19
• if you have a large/wide layer in your NN then you are basically turning the lower dimensional input into a higher dimensional feature layer... just wondering what kind of transformation you exactly have in mind
– oW_
Jan 13 '20 at 17:23
• I don't, hence the question ;) I've never trained encoder myself. Just wandering if I can use correlation mapping somehow to train encoder. Jan 13 '20 at 17:37

You could do that with a neural autoencoder using a custom loss function. Use a hidden layer, let's call it $$l_{encoded}$$, with more nodes than the features of the input data.
You have to code the custom loss: $$loss = corr(o(l_{encoded}))+MSE(o(l_{output}),\ input)$$,
where $$corr(o(l_{encoded}))$$ is the correlation of the output of the encoding layer and $$MSE(o(l_{output}), input)$$ is the mean squared error of the last layers output and the input instance.