Why would you want to decorrelated data?
As I am reading about PCA and whitening on image data for DNN, I wonder what is the purpose of achieving the identity covariance matrix in your data is?
Is it to remove interaction between variables, thus allow simpler models to express interactions without having to compute x1*x2?
Having only taken a few courses on this stuff, I'm going to offer my understanding on this.
The main reason seems to be simplifying, as you assume. It manifests in different areas though. One is mathematical - if your co-variance matrix is identity matrix then the math becomes much easier. The one directly following from this is computational - inverting a matrix is very costly, but identity matrix is its own inverse, so that's perfect.
Another part of the simplicity is definitely the ease of interpretation. Independent variables are easy to explain to a non-technical person, but as soon as you get into covariances, that goes out the window.
