5
$\begingroup$

I want to use feature selection and observation subsampling on my data, for several reasons:

  • feature selection for the usual motivations (reduce noise, decrease running time, etc.)
  • observation subsampling because I have strongly imbalanced data, and I want not to introduce bias towards the most prevalent class in downstream classifiers

My question is: is there a specific order in which I should do feature and observation selection? E.g. first feature selection then subsampling?

$\endgroup$
1
$\begingroup$

In my opinion, the methodological correct way to do it would be to first randomly select the observations (e.g., with stratified random sampling to sustain the class balances as in the original data) and then do any model building (feature selection is a part of model building) based on the sected examples.

You should asses the quality of your model with remaining observations that have not been used for either selecting features or training a model.

Hope this helps.

$\endgroup$
0
$\begingroup$

Here is how I would think about this problem.

If I select my features first, I then need a large enough subsample so that the variance of the model is not raised counter productively. That raises a few important questions.

  1. Can I get a large enough subsample that meets my unbiased criteria?
  2. Can I get a small enough subsample that meets my efficiency constraints?

If I select my subsamples first, I then need features that are useful for this subsample and I ultimately need to cover the whole feature space. That raises a few questions.

  1. Can I get an appropriately sized feature space for my subsample size? At the very least I would want a feature space that is an expert in some specific region, with the intent to cover the whole feature space with subsequent subsamples and feature selection.
  2. How many subsamples can I process while staying in my efficiency constraints?

I would choose my methodology based on the positively answered questions above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.