I am trying to solve a problem predicting a value between a range for a sentence:

The dataset looks like this:

Index_no               text_sentence                             value 

01                     yes I like riding a bike. 
                       I was 4 when I learned                   4.2311
                       to ride a bike. the colors of my 
                       bike is yellow and black.   

02                     i like riding my bike, i learnt 
                       riding a bike when i was 8 or            -2.11
                       9 years old ,my bike is sparkling 
                       pink with white marks  

Range of values is -7 to 7. Now, I am thinking about using a LSTM for text, but I am confused about the continuous output.

I was thinking about two methods:

Converting (normalizing the data between 0 and 1) and then after getting the output from network, denormalize the data, will this work?

Second approach, using a custom activation function?

Or how can I get output between a range?

  • 2
    $\begingroup$ The question is: what makes you believe that the LSTM will output categorical values by default :) ? $\endgroup$ – pcko1 Jun 19 '18 at 13:17
  • 1
    $\begingroup$ This is a difficult question to answer as written because it's hard to tell what you're looking for from this value or where it came from in the first place. There are a number of ways to get network output and put it in a range from -7 to 7. What is the distribution of the target values in the training set? Should your predictions be normally distributed around 0? Uniformly distributed over the whole range? I think it's worth getting an understanding of what properties the output should have before you come up with the method. $\endgroup$ – Matthew Jul 19 '18 at 15:05
  • $\begingroup$ Hi @AyodhyankitPaul, did any answer below help? $\endgroup$ – Escachator Sep 5 '18 at 15:38

Using Sigmoid Activation function (standard, no need for customization), you will get outputs in the range [0,1]. If you pre-scale your labels accordingly to fit in [0,1], the network will output values which you can manually re-scale to match the original amplitude. Hope it helps :)


Just take the last activation of the RNN and sum it. Then train using a loss suitable for regression, e.g. MAE or Quadratic Loss where you compare the real value vs your sum from the model.

You can also pass the last activation through a linear layer with an output of dimension one, for example.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.