Model building with neural networks

Question

Assume the existence of a collection of physical parameters and a collection of output variables which may depend on the physical parameters. An example in the training dataset consists of a vector with the actual measurement of the physical parameters, and another vector with the output values which were measured.

To my understanding, I can train a neural network such as a MLP to learn the mapping from the measured input parameters to the measured output values. If a model could be learned and assuming the learned relations are due to causality and not some unfortunate correlation, then for an arbitrary input vector the model could predict the vector of output values. Note that the relationship between input and output is not a 1:1 map, rather n:1. Different parameter choices may thus lead to the same output.

I want to go one step further and define an interval for each output value. I then want to know how I must set the ideal input parameter values to ensure that the output values are in the corresponding intervals, i.e. how to alter inputs for the output values to be within the ranges. I tried feeding the output values as input and trying to predict the physical parameters, but I have problems due to the ratio input/output. I have 130 physical parameters and only 10 output variables. How can I achieve this with a neural network?

Answer 1

I would assume that the physical input parameters occur in some sort of sequence & the Neural Network can easily identify patterns from it.

I would suggest you do some scaling on the output parameters. Or use non-linear activation functions so that your output ranges from upper bound & lower bound of that function.

Also i would like to suggest that you use LSTM. Recent time-series application using LSTM have promising outputs. Their application ranges from text generation to pixel filling in images & videos. They can be used to get bounded outputs & using torch neural networks allow you to implement them as per your requirements.

You can find implementation guide for Neural Network using torch library here

Answer 2

You can train the inputs using backpropagation, using almost the same formulas that train the first weight layers. Define a cost function based on how far each output is from one of its boundaries, calculate the gradient and minimize the cost. Starting from different random input states, this process will take the inputs to different points on the input boundary that you’re trying to compute.