How to perform a reggression on 3 functions using a Neural Network

Question

I am currently building a neural network using Keras to perform a regression.

I have 4 independent variables W,X,Y,Z. They are used to predict 3 different functions f1(W,X,Y,Z), f2(W,X,Y,Z), f3(W,X,Y,Z).

Should my output layer have 1 or 3 neurons? Also, should I be using a relu or linear activation function for the output layer? I'm currently using MSE for my loss function and adam for my optimizer.

For my metrics, should I use 'accuracy' or 'r2'?

Any suggestions? I'm sorry, I'm new to deep learning...

Answer 1

Should my output layer have 1 or 3 neurons?

The easiest thing to do is to create 3 separate networks, one for each function you want to approximate.

While it may certainly be beneficial if you combine all 3 outputs into the same model via a multi-task framework, I'd suggest starting off with separate networks, which is more intuitive. Then, if you want to see if combining them under one network improves your performance, you could check out how to create models with multiple outputs in keras.

Also, should I be using a relu or linear activation function for the output layer?

Linear, definitely.

For my metrics, should I use 'accuracy' or 'r2'?

Accuracy is a metric for classification, not regression. $r^2$ can be used, but for certain problems it isn't very reliable. You could try $r^2$, as well as MSE, MAE, MSLE (MSE of the $log$s of the predicted/actual values), etc.

Answer 2

Create a different networks for each function, each with a single output.

Ensure you have at least one hidden layer in each network. Use ReLU in this layer. Use a linear layer at the output without any activation function.

Since you are performing a regression, use the mean squared error to evaluate your networks https://en.wikipedia.org/wiki/Mean_squared_error.