I’m doing a project in which the suitability of Neural Networks (NN) shall be assessed and I am fairly new to the subject. More precisely the NN should replace an analytical model. This model calculates forces and torques applied on a structure based on displacement data. The task of the neural network is to approximate forces/moments via regression.

Some Authors state that a neural network consisting of input, one hidden layer and output layer might is sufficient for this purpose. Therefore I constructed a NN with these characteristics. The NN has 12 inputs, 8 hidden and 6 output neurons. The 12 inputs represent the deformation in two directions measured on 6 different points. The 6 Outputs represent the applied forces and torques (Fx,Fy,Fz,Mx,My,Mz).

For training and validation of the model 52 measurements, each one consisting of 12 deformations and 6 forces/torques have been used. Afterwards the data was extended to 174 measurements. In both cases the NN could not generalize well and the training “loss” was huge. I tried changing various parameters of the neural network (such as number of hidden neurons, activation function, optimizer, scaling of the data…). None of this led to any significant change. Afterwards I tried to reduce the complexity of the problem by reducing the number of output variables. Still with little success.

My main concern is that the amount of data available for training is way too small. Do any of you have any tips or suggestions? Could anyone verify my concern?

  • $\begingroup$ I would agree with your assessment regarding the data availability, using less than 200 samples to train a neural network from scratch will be very difficult. Usually more conventional statistical methods work better in situations where you have little data available. $\endgroup$
    – Oxbowerce
    May 19, 2021 at 9:16

1 Answer 1


If the training loss is “huge” it suggests that the is a bug in the model or the model is not sophisticated enough.

Do not confuse training loss with generalisation. Generalisation is measured by accuracy on the test set.

With regards to training data have you tried data augmentation? Can you apply transformations to the training data to derive new samples. E.g if I train a model to drive a car based on video input and control output the video and control inputs can be flipped to turn left turns into right turns?

  • $\begingroup$ Thank you. Your answered has helped. Now Im trying data augmentation. Hopefully I will get some results soon $\endgroup$
    – JAL
    May 21, 2021 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.