0
$\begingroup$

Can Transformer Models be used for Training Chatbots?

Note - I am talking about the transformer model google released on the paper 'Attention is all you need'

$\endgroup$
1
  • $\begingroup$ Please, consider upvoting the answers if you found them useful, and marking one of the them as correct if deemed so. Alternatively, please considering describing what the answers are lacking or why you think they are not correct, so that they can be improved. $\endgroup$ – noe Jan 9 at 16:06
0
$\begingroup$

A transformer is just a neural network. Sure, it is way more complex than a feed forward one, but it still is a neural network. As long as you provide the correct dataset (supervised case, so the correct pair input-target) your model should be able to learn the hidden representation that make it able to answer to new questions.

I found this interesting tutorial, try to check it out.

$\endgroup$
0
$\begingroup$

The Transformer model is a sequence-to-sequence model, that is, it is meant to address problems where the input is a sequence of discrete tokens (i.e. text) and the output is also a sequence of discrete tokens.

Therefore, a Transformer is well suited to be trained with a dataset of dialogs where the input is a statement or question and the output is the answer. This is usually called a "chit-chat" chatbot, because they are not backed by a knowledge base. They can just have "small talk".

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.