0
$\begingroup$

Is anyone aware of any successful implementation of reinforcement learning for NLP. I am looking to for chatbots which can learn automatically.

Tried searching internet but found very few articles like Reinforcement Learning For Natural Language Processing - Medium or papers like A Survey of Reinforcement Learning Informed by Natural Language.

But none of them provides a robust code which shows that it is working. Kindly suggest.

Improve this question
$\endgroup$
0
$\begingroup$

That's because using NN for chatbots have proven super challenging. Basically once you type a query the NN (whether its NN or RL) has to tag it to a particular intent based on which u give a templatized response ( generating a human like response goes into even more complex territory of natural language generation)

The issue with NN like LSTM , GRU etc is that they don't understand "turns" in a conversation. So if u type in a query one after the other the bot will treat them as separate sentences and try and tag them individually. But a human would form a context based on the whole conversation and can handle turns. There's some labelled data for a few verticals that shows turns in the conversation but it's just not enough

Coming to RL , it seems like a good fit for handling conversation since the current state (sentence) totally depends on the prior state. But what kind of reward function do you have in mind ? How, based on the action of the agent, will the network learn a policy to identify a "conversation"? When will u stop the episode ? When a conversation ends ? Or a fixed number? Me thinks these are still hard problems to solve since convergence is a challenge in such use cases

If someone has cracked it, they most likely will not publish anything since a true researcher will only publish stuff that is reproducible or they are just being secretive :)

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for reply. 2 comments here - 1. Intents may not be required. Intents are used for classification. These can be done using encoder-decoder scenarios e.g. autoencoders etc. Do yo have a different opinion on this or different understanding of applications of these architectures? 2. Are you confirm that bots do not exist who learn themselves (like using RL or some other continuous learning technique) ? $\endgroup$
    – Sandeep Bhutani
    Nov 28 '19 at 9:25
  • $\begingroup$ Also, If i remember correctly Tay bot by Microsoft learnt from community in real time. I am wondering what kind of learning it was - socialhax.com/2016/03/24/… $\endgroup$
    – Sandeep Bhutani
    Nov 28 '19 at 9:38
  • $\begingroup$ u make a good point regarding enc-dec but, afaik you need tons of data for a good enc-dec system in text (think machine translation); not sure if u will have that kinda data for conversations no ? thats why i hinted at intents (i come from the enterprise nlp space where its better to give templatized responses based on the intent of the query rather than depend on NLG) ..the MS bot was turned into a racist xenophobe mostly due to some enc-dec system that did online learning ..again, just my opinion..please don't crucify me for it :) $\endgroup$
    – Vikram Murthy
    Nov 28 '19 at 17:11
  • $\begingroup$ but then even if one were to use enc-dec system, how would u define the rewards for the agent ? in systems like VAE /AE for image corrections, at least you can check the distance between input image and the decoded image generated since both of them will be EXPECTED to have similar distributions. I can't imagine something like that for text especially when you are inputting parts of a conversation. Any thoughts on those ? just curious $\endgroup$
    – Vikram Murthy
    Nov 28 '19 at 17:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.