Training a stateful LSTM model on sequences of varying lengths can potentially lead to differences in the final state of the network, as you have observed. This can be caused by the fact that the network's internal state is updated at each timestep, and the number of timesteps that the network processes for each sequence will depend on the length of the sequence. As a result, the final state of the network may be different for sequences of different lengths, which can affect the performance of the model.
To mitigate this issue, you can try using a fixed-length sequence for all of your training data, by padding the shorter sequences with additional timesteps of zeros or some other placeholder value. This will ensure that all of the sequences have the same length, and the network will process the same number of timesteps for each sequence. This can help to ensure that the final state of the network is consistent across all of your training data, and can improve the performance of the model.
Another option is to reset the state of the network after each sequence, instead of only resetting it after all of the sequences in a batch have been processed. This will ensure that the network's state is reset at the end of each sequence, regardless of its length, and the network will start each new sequence with a consistent initial state. This can help to prevent differences in the final state of the network for sequences of different lengths, and can improve the performance of the model.
Ultimately, the best approach will depend on the specific characteristics of your dataset and your goals for the model. You may need to experiment with different techniques and configurations to find the best solution for your problem.