1
$\begingroup$

I am working on a financial predict problem. which means it is a time series prediction problem.

I have three features, which have high correlation(each two's corr is about 0.6) And I do the linear regression fit.

I assume that the coefficient should be similiar among these three features, but i get a coefficient vector like this:

[0.01, 0.15, 0.01]

which means the second features have the biggest coff(features are normalized), and it can dominant the prediction result.

I dont know why. I think adding weak features can boost my prediction model, but i think the second feature is dominant in my model, and other features are worthless.

Why one of features can be dominant in the model, did I miss something?

$\endgroup$
4
  • $\begingroup$ What is the scale of the $x$? Standard deviations or "real" units? Did you scale the data? $\endgroup$
    – Peter
    Commented Apr 4, 2021 at 11:12
  • $\begingroup$ @Peter yes, i scale the data by standardize method: x = (x-x.mean())/x.std() $\endgroup$
    – nick
    Commented Apr 4, 2021 at 14:06
  • $\begingroup$ correlation(each two's corr is about 0.6) ? Do you mean -Is it for each of three correlations ? $\endgroup$ Commented Apr 6, 2021 at 15:01
  • $\begingroup$ Is there regularization being used? What tool/API/parameters are you using? $\endgroup$
    – Craig
    Commented Apr 7, 2021 at 10:44

1 Answer 1

1
$\begingroup$

The first thing that comes to my mind is that you might have not normalized your features correctly. Generally a feature ranging between a bigger range of values, compared to the other ones, is going to be more influencial in terms of the models' output.

In order to midigate this issue, one common practise is to transform your features into having zero-mean and a variance of one. In this way, it is guaranteed that all your features have identical range of values.

Other than that, It may be just that the dominant feature is indeed more indicative for your time series prediction and thus your model has learned to rely its predictions on this specific feature.

$\endgroup$
2
  • $\begingroup$ thanks, i think i have normalized data. maybe it is the best for prediction. but the similiar low correlation with target make me confused, is there any method can know the importance of features before fitting $\endgroup$
    – nick
    Commented Apr 4, 2021 at 14:10
  • $\begingroup$ what about the gold standard PCA? $\endgroup$ Commented Apr 4, 2021 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.