0
$\begingroup$

I have a dataset that contains n features scaled between [0,1]. I would use an unsupervised feature selection algorithm (variance thresholding). How can I compute the threshold value?

$\endgroup$

1 Answer 1

0
$\begingroup$

Variance thresholding is used to select those features with a variance above the suggested threshold. Ideally you would want to take in all features with a non-zero variance but I'm not sure of the data youre handling, it would be better to calculate the variance of the individual feature, arrange them in the increasing order of variance and then select that value where the variance sharply increases.

Or you could do PCA and find the order of the importance of features and then set the threshold.

$\endgroup$
2
  • $\begingroup$ I use threshold=0.0 after that print(selector.variances_) [1.05101703e-03 6.82049836e-03 2.38980022e-03 8.45874708e-04 6.62252127e-03 2.65356046e-03 3.43471582e-04 6.33013082e-03..............] $\endgroup$
    – Ramzi
    Nov 13, 2020 at 22:44
  • $\begingroup$ and max variance equal to 1.39243526e-02 then could I use threshold=0.01? $\endgroup$
    – Ramzi
    Nov 13, 2020 at 22:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.