0
$\begingroup$

I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100). The target values are scaled by three methods.

  • method 1 : logarithmic scale
  • method 2 : MinMaxScalar
  • method 3 : divided by 1100

But these methods could not succeed to get the better result. The code is not able to learn and predict well. Specially in small values of the target and got minus values as the prediction.some useful information as follow;

model = Sequential( )
model.add(InputLayer(input_shape= (14, )))
model.add(Dense (14,'tanh' ))
model.add(Dense (5,'tanh' ))
model.add(Dense (3,'tanh' ))
model.add(Dense (1,'linear' ))

Loss : MSE, optimizer : adam
thank you in advance for your attention

Improve this question
$\endgroup$
3
  • $\begingroup$ You could also try a StandardScaler / z-Normalization. But without knowing the use case, it is hard to say if this would make sense. If all target normalization methods fail to give good results, I would search for the problem elsewhere. Do you have enough training data? Did you try different network architectures? What about the activation functions? If you want to have non-negative output, try a relu-output-layer. If the output should be between 0 and 1, try a sigmoid layer, or scale it between -1 and 1 and use tanh. $\endgroup$
    – Broele
    Commented Aug 20, 2023 at 11:19
  • $\begingroup$ The distribution of the input features could also be part of the problem. Look at a histogram of each feature - if a feature is highly skewed, multimodal/mixed, that could be causing issues and would benefit from transformation into a more uniform distribution. matplotlib.pyplot has a .hist plot, and pandas also has a .hist, which are useful for making quick histograms. $\endgroup$
    – MuhammedYunus
    Commented Aug 20, 2023 at 11:23
  • $\begingroup$ Thank you for your reply. The training data numbers are 194 and it is a big problem. Anyway, I have tried different network architectures and the best option for hidden layers has been 14, 5, 3, 1 with tanh as the activation function. the feature values are in range of (-1 , 1). so I did not use sigmoid. $\endgroup$
    – Mali
    Commented Aug 20, 2023 at 13:35

1 Answer 1

Reset to default
0
$\begingroup$

Sometimes, using a statistical learner, such as support vector regression, is worth a try. If you encounter similar problems there, that might be another hint that the problem is located with the dataset. Using principal component analysis to understand the dimensionality of the problem can also be useful. A slightly different approach for understanding how the features are related tp the target is to use the permutation feature importance. From this, you can find if some of the feature are redundant.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.