I have time-series data obtained from a video. The data is composed of bitrate and corresponding label pairs for each timestamp:


The distribution over the first 30 seconds is as follows:

data distribution

I have built an LSTM model for this dataset to be able to classify the labels based on the bitrate. However, it seems that my model is not able to learn. Validation accuracy starts from approximately 0.3 (makes sense, since I have 2 classes (log2 = 0.3)) and it does not improve.

Do you have any idea about this? Is it normal considering this sample data distribution, or is something might be wrong with my model? Thanks!

  • $\begingroup$ maybe it's an issue of putting softmax cross entropy right after LSTM. There should be a dense-layer (non-activated) in between the two. datascience.stackexchange.com/q/25525/43077 $\endgroup$
    – Kari
    May 22, 2021 at 14:44
  • $\begingroup$ Hey @Kari thank you for the response! Frankly, I am following the same logic you have mentioned but I am not sure how to integrate the Dense layer here. I just put the core part of my model below: paste.ubuntu.com/p/bhrHCqpbmD Can you help me to try your idea? $\endgroup$
    – bbasaran
    May 22, 2021 at 15:20
  • $\begingroup$ Sorry for the mistake in my last comment. In case you found a chance to have a look at my code, you can see that the LSTM output is first forwarded to "Linear" layer, and afterwards, it is forwarded to "CrossEntropyLoss". So I guess there is already a dense layer ("Linear" in Pytorch) between them as you have said. $\endgroup$
    – bbasaran
    May 22, 2021 at 19:32
  • $\begingroup$ What's the meaning of the label? Is it generated by a process that has any relation to the bitrate? $\endgroup$
    – Paul
    May 23, 2021 at 19:32
  • $\begingroup$ Hey @Paul . The label is the class of stages in a video. You can imagine that 0 stands for walking, while 1 is for running. My script manually labels those stages, and my goal is to find a relation between the instant bitrate and those stages. My model should be able to predict the label just by looking at the instant bitrate of that moment. $\endgroup$
    – bbasaran
    May 23, 2021 at 19:55

1 Answer 1


An LSTM uses the preceding sequence of bitrates to produce a label. If the label can be predicted from those bitrates, then your problem is learnable with an LSTM.

From your comment: The label is the "stage of the video": Running, walking, etc. The question is if this can be derived from the bitrate? Walking with a busy background might yield a higher bitrate than running. Judging your example by eye, I suspect it cannot. You need to get more features from the video than just the bitrate, I think.

  • $\begingroup$ Exactly. I think in the same way. However, it seems that my model cannot learn for now. This brings me this question: is my dataset not consist of any pattern to be learnt or should I blame my model for this? $\endgroup$
    – bbasaran
    May 23, 2021 at 19:59
  • $\begingroup$ It could be solved with an attention layer. Because the attention layer will make sure that the model is only looking into bitrate and not the noise. $\endgroup$ Jul 19, 2023 at 8:15
  • $\begingroup$ That is incorrect. An attention mechanism has no such guarantees and there is also no reason to think that it would help. $\endgroup$
    – Paul
    Jul 20, 2023 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.