# Clipping threshold of softmax layer

I recently came across a paper on using (rather simple version of) LSTM for sentiment classification, and it describes its network settings as:

We randomize the parameters with uniform distribution U(-0.003, 0.003), set the clipping threshold of softmax layer as 200 and set learning rate as 0.01.

I am trying to reproduce their results with my code written in Tensorflow. I have to say I find it a bit confusing by the meaning of "set the clipping threshold of softmax layer as 200". 200???

Can someone explain this to me please so I know how to implement this using Tensorflow? Thanks so much!!!

Update, ok so I found the their code and below is how they use this clipping threshold of softmax layer:

for(int k = 0; k < softmax.outputG.length; k++)
softmax.outputG[k] = 0.0;
softmax.outputG[goldPol] = 1.0 / softmax.output[goldPol];

// if ||g|| >= threshold, then g <- g * threshold / ||g||
if(Math.abs(softmax.outputG[goldPol]) > clippingThreshold)
{
if(softmax.outputG[goldPol] > 0)
softmax.outputG[goldPol] =  clippingThreshold;
else
softmax.outputG[goldPol] =  -1.0 * clippingThreshold;
}


I still can't say I fully understand the use of this threshold..

• You probably reading this one. One thing I can think of – they mean softmax with temprerature. Although they give the definition of softmax they are using which is without temperature parameter... Confusing! Commented Dec 23, 2016 at 0:01
• I updated my original post with some of their code that I found btw.. Commented Dec 23, 2016 at 14:00