I am working on a financial predict problem. which means it is a time series prediction problem.

I have three features, which have high correlation(each two's corr is about 0.6) And I do the linear regression fit.

I assume that the coefficient should be similiar among these three features, but i get a coefficient vector like this:

[0.01, 0.15, 0.01]

which means the second features have the biggest coff(features are normalized), and it can dominant the prediction result.

I dont know why. I think adding weak features can boost my prediction model, but i think the second feature is dominant in my model, and other features are worthless.

Why one of features can be dominant in the model, did I miss something?

  • $\begingroup$ What is the scale of the $x$? Standard deviations or "real" units? Did you scale the data? $\endgroup$ – Peter Apr 4 at 11:12
  • $\begingroup$ @Peter yes, i scale the data by standardize method: x = (x-x.mean())/x.std() $\endgroup$ – nick Apr 4 at 14:06
  • $\begingroup$ correlation(each two's corr is about 0.6) ? Do you mean -Is it for each of three correlations ? $\endgroup$ – Subhash C. Davar Apr 6 at 15:01
  • $\begingroup$ Is there regularization being used? What tool/API/parameters are you using? $\endgroup$ – Craig Apr 7 at 10:44

The first thing that comes to my mind is that you might have not normalized your features correctly. Generally a feature ranging between a bigger range of values, compared to the other ones, is going to be more influencial in terms of the models' output.

In order to midigate this issue, one common practise is to transform your features into having zero-mean and a variance of one. In this way, it is guaranteed that all your features have identical range of values.

Other than that, It may be just that the dominant feature is indeed more indicative for your time series prediction and thus your model has learned to rely its predictions on this specific feature.

  • $\begingroup$ thanks, i think i have normalized data. maybe it is the best for prediction. but the similiar low correlation with target make me confused, is there any method can know the importance of features before fitting $\endgroup$ – nick Apr 4 at 14:10
  • $\begingroup$ what about the gold standard PCA? $\endgroup$ – Yiannis Ath Apr 4 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.