Problem of finding best combination of features when desired feature is feature some_feature_A/some_feature_B

Question

Problem is stated: we have giant csv file with one target column and rest are inputs, we don't know these features impact target but we would like to use algorithm that besides using linear and non-linear transformations will also take into account that right solution would be some_feature_A/some_feature_B. Is there algorithm that will allow to take this case into account? One way is to craft these feature columns yourself but is there better way?

Answer 1

In theory, I think a deep neural net might be able to find features that are the product of two other columns. There exist some nice mathematical results which guarantee the ability of a neural net (with certain activation functions) to approximate any function, so there's no theoretical reason why a neural net couldn't compute the division function $f(x, y) = \frac{x}{y}$. That being said, it may be difficult to achieve in practice without a little preprocessing.

If you want to give it a try, I would suggest adding additional features obtained by taking the logarithm of any numerical features in the dataset. This might make it easier for the network for learn features that are products of other features (since $\log{(x * y)} = \log{x} + \log{y}$ and $\log{x / y} = \log{x} - \log{y}$).