# Can we learn f(x)=1/x using a neural network exactly?

Is there a way to train a neural network as $$f(x) = {1 \over x}$$ precisely?

• – Erwan Sep 7 '19 at 19:33
• On what domain? Certainly (using any of the usual activation functions) a neural network's overall function is continuous, so you can't hope to even properly approximate 1/x over an interval containing zero. I'd doubt you can get the function exactly (again using the common activators) on any interval, but I don't know how to prove that negative assertion. – Ben Reiniger Sep 8 '19 at 4:10
• The universal approximation theorem says that a neural network can approximate a continuous function on a closed and bounded set of the real line. Therefore, let’s restrict the question to a closed interval. If that interval does not contain 0, then the universal approximation theorem says that we can get arbitrarily close to $y=1/x$. If that interval does contain 0, then the function is not continuous on the whole interval, so the universal approximation theorem does not guarantee the ability to approximate at 0. – Dave Feb 7 '20 at 17:18

• @NagabhushanSN The Kolmogorov's theorem that you have read about says that any function of $n$ variables can be approximated with a sum of functions of 1 variable. This is how a neural network is constructed. It is a combination (a sum) of neurons (functions of 1 variable $z=\sum w_i x_i$). The theorem doesn't say which functions should be in the neurons: sigmoids, ReLu, or other. I can approximate any function with $1/x$-neurons. If your target function is similar to ReLu, you need just a few ReLu neurons to approx it. If i is very different from ReLu, you need a lot of ReLu neurons. – Vladislav Gladkikh Oct 26 '20 at 7:34
• $x=0$ is not in the domain of $f(x)=1/x$, so you wouldn’t need the gradient at $x=0$, but you may need the gradient arbitrarily close to zero. – Joe Jun 7 '20 at 15:38