0
$\begingroup$

I would like to work out an estimate for some variable Xi using values of a bunch of other variables Xj, j in Y (I guess this should include Xi too) and how correlated they are to Xi.

So if I do weight(j)=correlation(i,j)/sum over Y of other correlations, then this sum can be zero but I feel that might not be the only problem. I could use the min-max remapping but then the most negatively correlated variables will have no or little effect, whereas in fact they should have a strong negative effect.

One obvious fix would be to ABS() the sum but I could not prove to myself that this is the correct thing to do. It may be possible that each correlation needs to ABSed in the sum (then it would resemble the way stdev is calculated - then do I need to square my correlations?).

I simplified the problem I am trying to solve for this question but I am not sure if the details would affect the solution. There is a time dimension to it, and I am trying to use changes over each variable over time and their correlations to work out the estimate for the next change or a chain of changes I then multiply with the current value. (I don't know if this step is necessary or if proper normalisation of correlations would correct for this even if I used absolute values and not relative changes).

$\endgroup$

1 Answer 1

1
$\begingroup$

The appropriate method, if you have the data to use it, is linear regression. Simply using correlations fails to take into account that the other variables are also correlated with each other; for example, any averaging method like the one you describe will overweight any duplicated variables.

$\endgroup$
1
  • $\begingroup$ Could you elaborate a little please, correlations in my case are actually functions (of time shift), not values. So if we have for all i in I, Xi is a correlation function to some X0, where Xi(t) shows how correlated X0(w) is to Xi(w+t), then to do linear regression we need I*T dimensions, which I am not sure is possible. I am probably wrong and would love to know why. I completely agree with the last point about giving too much weight to duplicates but I don't quite get the proposed alternative method. $\endgroup$
    – rudolfovic
    Jun 16, 2016 at 4:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.