I am doing a prediction on a data set where labels have positive values (time values). After training a simple Linear pytorch model I get negative values for time despite being 0 negative values in the training set. Is there some way to punish the model hard for making negative predictions so that the model would understand to make positive predictions (because taking ReLU does not feel correct to me)?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
If you know that your output are positive, I think it makes more sense to enforce the positivity in your neural network by applying relu function or softplus $\ln(1. + \exp(x))$. You could also have a look at Generalized models which extend linear regresssion to cases where the variable to predict is only positive (Gamma regression) or between 0 and 1 (logistic regression). If you are predicting a categorical variable, you could also perform one hot encoding and transform your regression problem in a classification. Last but not least, as suggested in the last question it might be interesting to normalise your output between 0 and 1 and have a logistic regression in the last layer. Hope this help
$\begingroup$
$\endgroup$
My understanding is that the Linear module is essentially linear regression. So if you say “simple Linear” I assume if there is any significant slope, then going negative is somewhat unavoidable if your testing data is beyond the range of the training data. I say all that really to say, perhaps the Linear model is not the best choice.
Regarding ReLU, it might not make much difference as assuming you are using sigmoid activation. Both return a value between 0 and 1.
The other thing you should consider is whether to normalize your input between 0 and 1. HTH
