1
$\begingroup$

Problem is stated: we have giant csv file with one target column and rest are inputs, we don't know these features impact target but we would like to use algorithm that besides using linear and non-linear transformations will also take into account that right solution would be some_feature_A/some_feature_B. Is there algorithm that will allow to take this case into account? One way is to craft these feature columns yourself but is there better way?

$\endgroup$

1 Answer 1

1
$\begingroup$

In theory, I think a deep neural net might be able to find features that are the product of two other columns. There exist some nice mathematical results which guarantee the ability of a neural net (with certain activation functions) to approximate any function, so there's no theoretical reason why a neural net couldn't compute the division function $f(x, y) = \frac{x}{y}$. That being said, it may be difficult to achieve in practice without a little preprocessing.

If you want to give it a try, I would suggest adding additional features obtained by taking the logarithm of any numerical features in the dataset. This might make it easier for the network for learn features that are products of other features (since $\log{(x * y)} = \log{x} + \log{y}$ and $\log{x / y} = \log{x} - \log{y}$).

$\endgroup$
2
  • 1
    $\begingroup$ ok thanks to your answer I found solution some_layers -> log -> some_layers -> exp_activation -> some_layers and voila $\endgroup$
    – quester
    Jul 11, 2019 at 14:56
  • $\begingroup$ Nice! good idea $\endgroup$
    – zachdj
    Jul 11, 2019 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.