1. Should we convert our inputs to on-hot-vectors and expect one-hot-vectors as output? I mean can we feed LSTM with a vector like x=[12, -234, 54 , 78 , 12 , 6], and have a label vector like this: y=[13, -230, 50, 80 , 9 , 7]? (And we don't use one-hot-vectors at all). Will such network work properly? Or it's better to convert inputs/outputs to a one-hot-vector and this is essence of LSTM?

  2. If feeding LSTM with one-hot-vector isn't a necessary rule, and we like to feed our network with such vectors in my previous question, should we again use softmax() function for out outputs? Or we can use better options for such problem(or even don't use any functions there)? If we must(or better) to use softmax, how can we interpret it's result?

  3. If it's better to convert our inputs/outputs to one-hot-vectors, can we use two or three hot vectors(I mean: x =[1,0,0,1,0,0] or x=[0,1,1,1,0,0])? Does this work properly or it disrupts the LSTM performance?

  1. This depends on what your data is representing and what you want to predict. My understanding of One-Hot-Encoding is that this should only be used for encoding of categorical features. For example, if you have a feature representing a category of K classes, you should one hot encode this as well as the Y variable (if you are trying to predict this categorical variable). Of course, have the final layer be a softmax to output a distribution of size K.

  2. This highly depends on what your data is representing. If categorical, see above. If just a simple numeric, you should not one hot encode this. You could, I guess, if this set of integers is finite and small, but there is no need to learn the extra weights. You should only be using a softmax when you want to output a vector of K dimension where the entries all sum to one (perfect for representing a probability distribution over K classes). The final layer should output whatever it is you want to predict. If you want to predict something as simple as a numeric variable at the next time-step, just have a dense layer of size 1, with some activation function (relu probably). More information about what exactly you are trying to predict is needed to know how to recommend anything concrete here.

  3. I'm not sure about this. You could have inputs and outputs represented this way, but you wouldn't use a softmax activation at the end. You would use a dense layer outputting a vector the size of your X variable, that is, if that's the variable you are trying to predict. I'm not super well trained in various types of use cases for LSTMs, but from my experience, I can't think of a reason to do this.

  • $\begingroup$ Thank you very much kylec, unfortunately I have not enough reputations to give you a +vote at this time. BTW, about your question in part 2, I am trying to do stock price prediction, and by [200 210 199 205 100 etc.] I mean [open high low close volume etc.] numbers. What do you think about it? I am confused that what input/output vectors should I have for this porpuse? Also what activation functions? $\endgroup$ – WDR Feb 21 '19 at 8:28
  • $\begingroup$ All good! I'm still a newbie to StackOverflow (well, actually participating, at least) so I'm restricted as well. I'm assuming all of your variables are numeric, so if you are trying to predict all these numerical features (essentially have a prediction for the entire next time-step), have your Y variable be the next time-step values for each variable, and at the end of your network have a Dense layer with units equal to however many numeric variables you are trying to predict. I'd start with a relu activation for this layer. $\endgroup$ – kylec123 Feb 21 '19 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.