Is it possible to feed embeddings from XLM- RoBERTa to transformer seq2seq model? I'm working on NMT that translates verbal language sentences to sign language sentences (e.g Input: He sells food. Output (sign language sentence): Food he sells). But I have a very small dataset of sentence pairs - around 1000. And the language is a low resource language.
I am a new researcher on the field of deep learning. Please help with your valuable advice.