2
$\begingroup$

So say you have some data that consists of some values:

1.3, 0.9, 1.1

You introduce a new feature which is the average of these values: 3.3

In this example lets say that you know the average of these features is a good indicator to classify data with. My question is that, you're not actually adding anything to the model by creating this 4th feature, all the data is already in the features so does having this average help the model training identify the relationship more easily?

$\endgroup$
1
$\begingroup$

It depends what model will you train and depend on the new feature. However, your answer is that yes. it is possible to help the performance of that model. For example, suppose the new feature is a non-linear combination of other features (multiple of the other feature) and your model is a linear classifier. When you add the new feature might help to classify better the data depends on the context.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I do not have enough reputation to simply add a comment. It will be worth to check the diferent methods for semi-supervised learning. It is currently a well established topic (there are examples back to the 2000s, maybe elder) principally adressed to clustering and data classification but there are examples about regression (https://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs169689). The question is a little bit vague so just make your own search and pick what you are interested in.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Yes, in principle this type of feature engineering can help a model and - if the transformation is well-chosen - almost always does in practice.

Some models may simply not be able to figure out the kind of transformation that you give them without your help (e.g. a model that uses all predictors only in a linear fashion will not be able to represent non-linear relationshps). But even those that can figure it out (e.g. xgboost or some deep neural network should eventually with enough data manage to represent arbitrarily complex functions of multiple variables) can be helped a lot (in terms of performing better / getting to the same level of performance with less data) by providing them a good transformation that they otherwise need to learn by seeing lots and lots of examples.

If you look at the description of their chosen approaches of winners of kaggle competitions, you will see that some kind of clever feature engineering often played a major role.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.