I have a problem where I want an LSTM to predict the resistance of a body. This value can also be calculated if we know the drag coefficient and the speed of that body. In my case, at inference time, the speed is known, meaning that I can do the following:
Which one should I use as my learning target?
Interesting question!
Experiment with both options and see which performs best!
predict the drag coefficient, and then calculate the resistance accordingly
If you do this, your network will try to optimize something different than your actual goal (which is the resistance). This means that your model will not "care" if the resistance you eventually calculate is any good, which can result in strange results.
predict the resistance directly
This would be better from a machine learning perspective as your model's goal will be the same as yours, however, you will lose the advantage that you have by knowing how the resistance is calculated.
Solution A
Predict both and then have a final step to decide what your final resistance will be. With LSTM, this is definitely possible, your target will just become 2 numbers instead of 1.
Solution B
The best solution, in my opinion, would be to have the LSTM output a single number (which would act as the drag coefficient), and then, add a layer which calculates the resistance using the known formula so that you can backpropagate on the entire thing, and you get the best of both worlds. In PyTorch this can be done rather elegantly. The big caveat is that the formula to calculate the resistance needs to be differentiable.
