1
$\begingroup$

Can feature engineering avoid overfitting? If yes, are there any relevant papers that state this?

$\endgroup$
3
  • $\begingroup$ Sure it can, just like throwing a coin into a well can. Maybe. $\endgroup$ Commented Mar 9, 2023 at 13:19
  • $\begingroup$ can you give me some reference? $\endgroup$ Commented Mar 9, 2023 at 13:57
  • $\begingroup$ Cross-posted on reddit. $\endgroup$
    – noe
    Commented Mar 9, 2023 at 15:13

1 Answer 1

2
$\begingroup$

Yes, it can be one of the ways to avoid overfitting. Overfitting occurs when a model becomes too complex and learns to fit the training data too closely, which can lead to poor performance on unseen data. One way to prevent overfitting is to reduce the complexity of the model, and feature engineering can help achieve this by reducing the dimensionality of the input space. This can be achieved by identifying and removing redundant or low-correlated features that contribute negligible information to the learning process. This can make the model more efficient by reducing the number of input variables, while still retaining important information that can be used to make satisfactory predictions. By doing so, the model can generalize better to new data and avoid fitting noise or irrelevant details in the training data.

It should be mentioned that feature engineering should be done carefully taking into account the specific characteristics of the data and the problem itself as it can introduce potential bias that can affect the generalization ability of the model. Some papers to refer to are:

Applied Predictive Modeling

The Problem of Overfitting

Handling over-fitting in test cost-sensitive decision tree learning by feature selection, smoothing and pruning

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.