-1
$\begingroup$

In a machine learning algorithm, I have a feature that has a value in the range 0-20 it is very rarely value goes over 20 and if does I clamp it 20.

Does it help the neural network model somehow using reducing the infinite floating number set to integers between 0-20? Or even further if I categorize between the floating numbers between like; 0-5 than 0, 5-10 than 1, 10-15 than 2, and 15-20 than 3 does it helps my model to converge better and be more accurate? Does it reduce the effect of "Curse of dimensionality" because the possible inputs are reduced from an infinite set to few possibilities?

$\endgroup$
1
  • $\begingroup$ The only way to know if it helps is to try. What you're describing is a method which sometimes make sense for traditional ML methods (depending on the data and the task, of course). However with neural networks you shouldn't do this kind of feature engineering, because they deal with features in a very different way (when designed properly). $\endgroup$
    – Erwan
    Oct 16 '20 at 23:06
1
$\begingroup$

This would not reduce the effect of the curse of dimensionality because you are not reducing any dimensions, simply the values of one dimension. A valid reason to do this would be if there are so few training examples above 20 that your neural network struggles to learn much about them. But as Erwan suggested, you should simply try clamping and not, and compare validation accuracies. I would suspect that a well designed neural network architecture could better use the information of all values from 0-20 and would not benefit from throwing away information by binning it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.