Censored output data, which activation function for the output layer and which loss function to use?

Question

I am building a neural network to solve a regression problem. The output is a single numerical value.

Unfortunately, the output is censored: the values below 0 were recorded as 0, and postive values remained unchanged.

What activation function should I use for the output layer (maybe ReLU)? How to define the loss function, should I just use RMSE? (because the output is censored, we want the neural network to be able to generate 0 output, and positive values).

Edit 1:

The problem is to predict the electricity demand time series based on multiple input variables. Only the values above a certain threshold are being recorded, hence the output is censored. We have a lot of numerical/categorical input variables: time of the day, air temperature, days in a week, holiday/workday, etc. We want to build a neural network model to predict electricity demand (0 or positive) based on the input variables.

Answer 1

We can't tell you what loss function to use. That is based on business needs. In particular, what is the cost of being wrong? Is the cost of being wrong proportional to relative error? absolute error? Something else? That will drive the choice of loss function. You should try to choose a loss function where the value of the loss function is proportional to the cost of the error (e.g., the monetary cost to your company).

Using ReLU as the activation function in the final layer would be sensible given that your data is censored to replace all negative outputs with 0.