1
$\begingroup$

Problem is stated: we have giant csv file with one target column and rest are inputs, we don't know these features impact target but we would like to use algorithm that besides using linear and non-linear transformations will also take into account that right solution would be some_feature_A/some_feature_B. Is there algorithm that will allow to take this case into account? One way is to craft these feature columns yourself but is there better way?

$\endgroup$
1
$\begingroup$

In theory, I think a deep neural net might be able to find features that are the product of two other columns. There exist some nice mathematical results which guarantee the ability of a neural net (with certain activation functions) to approximate any function, so there's no theoretical reason why a neural net couldn't compute the division function $f(x, y) = \frac{x}{y}$. That being said, it may be difficult to achieve in practice without a little preprocessing.

If you want to give it a try, I would suggest adding additional features obtained by taking the logarithm of any numerical features in the dataset. This might make it easier for the network for learn features that are products of other features (since $\log{(x * y)} = \log{x} + \log{y}$ and $\log{x / y} = \log{x} - \log{y}$).

$\endgroup$
2
  • 1
    $\begingroup$ ok thanks to your answer I found solution some_layers -> log -> some_layers -> exp_activation -> some_layers and voila $\endgroup$
    – quester
    Jul 11 '19 at 14:56
  • $\begingroup$ Nice! good idea $\endgroup$
    – zachdj
    Jul 11 '19 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.