2
$\begingroup$

what are the differences between feature weighting and feature selection? And is feature importance like feature weighting?

$\endgroup$
1
$\begingroup$

Feature selection is choosing among your features what are the most important ones.

Feature weighting is from the features that you have selected ranking the ones that contribute most to the model.

Just to keep a basic and simple response.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks Carlos Mougan. $\endgroup$ – lona Jan 8 at 5:17
0
$\begingroup$

TL;DR

Feature selection is a pre-processing step.

Feature weighting is a learned step.

Feature selection

This is a pre-processing step. Meaning that you choose which features the model even gets to see. This is important because in large enterprises, not all data is available from the same source or there is a cost to getting certain data. Therefore, features needed to be selected beforehand and effort is put to retrieve only the necessary data.

Examples:

  1. Medical Records might be useful for determining insurance premiums, but due to regulations you might leave that feature out.
  2. You are detecting topics of incoming e-mails. The past purchases of the sender might help to determine the topic in the e-mail but accessing the purchases database is more difficult than the e-mails, so you might choose to use only the e-mail texts instead.

Feature Weighting

This is a learning step. At this point, you can assume that feature selection is done, and that you have access to all the data you wanted.

Now the idea is to determine the importance of the feature coming to your model. This can be manually set, but ideally is based on some learned metric.

Feature importance is like feature weighting.

Example:

  1. You have an ensemble model, where all the feature coming into this model are actually predictions from other models. You might weight the predictions of these other models based on their individual performance. Then, your ensembler takes predictions from good performing models with more weight than from those with poorer individual performance.
| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks BrunoGl. $\endgroup$ – lona Jan 8 at 5:16
  • $\begingroup$ @Iona, glad to be of help. If you feel like your question has been answered. Feel free to mark one answer as accepted. $\endgroup$ – Bruno Lubascher Jan 8 at 9:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.