0
$\begingroup$

In the classical linear regression implementation, if I suspect the square of the values of the column is correlated to the target, then I actually need to create a new column with the squares for the algorithm to make use of that.

Is this also necessary when using neural networks? I know it's a broad question - are there cases where this is necessary and cases where it isn't?

$\endgroup$
2
$\begingroup$

You don’t necessarily need to, according to the universal function approximation theorem.

  • It is easier for a neural network to learn an identity function than some other function, so if one of the inputs definitely needs to be squared your network will learn faster if you pass the input already squared
  • If your network is sufficiently large it should work out that squaring that input is helpful and approximate the squaring function as part of the overall learning process
| improve this answer | |
$\endgroup$
  • $\begingroup$ Well, for this particular problem passing the squared column made a HUGE difference :), but I believe it was because by adding an additional column the complexity of the function increased (by 1 dimension) $\endgroup$ – Guillermo Mosse Aug 29 at 7:08
  • $\begingroup$ It likely meant that the neural network could focus on learning to approximate the other bits of the function you wanted it to learn, rather than also learning how to square that particular input. You might also consider passing the unsquared version of the column as well as the squared one, and let the neural network decide how to combine them. $\endgroup$ – Nicholas James Bailey Aug 29 at 7:16
  • $\begingroup$ That's what I did, actually: I passed both. $\endgroup$ – Guillermo Mosse Aug 29 at 8:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.