1
$\begingroup$

I want to train a deep neural network to solve a regression problem with target values in range $[-1,1]$. The problem is that most of these target values are really close to 0 (like $10^{-5}$), but there are also some values (a really small fraction) that are larger (closer to $1$ or $-1$).

What is the best way to handle this problem?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

EDIT: updated to account for potential $log(0)$ error

You can convert the answer into log space. If you take the output and then just take the log it will radically reduce the difference. If before your regressor outputs some number from input data $x$:

$$y = f(x)$$

Then you can just take the log of the output with a constant added to avoid $log(0)$ error:

$$y = log(f(x)+c)-c$$

Alternatively, you could just "bin" the output and make it a classification problem- perhaps a new class for every $0.1$ range. This will make the problem much easier to train, but of course is only doable if you're ok losing that precision. Combining log space with classification are also not mutually exclusive.

Improve this answer
$\endgroup$
2
  • $\begingroup$ $\log(x)$ is not defined for $x\le 0$. $\endgroup$
    – Dave
    Nov 18, 2022 at 18:20
  • $\begingroup$ True, but you can still avoid that by adding a constant value to the output and then re-normalizing $\endgroup$
    – Dan
    Nov 18, 2022 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.