I’m very new to the field of deep learning. My aim is to make a translation between Catalan to Catalan Sign Language. The grammar of the two languages is different
Input: He sells food. Output (sign language sentence): Food he sells.
I've been playing around with XLM-R and go the token id like this
input Ids: [200, 100, 2003, 1037, 3835, 3351, 5012, 300, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
I don't know how to use the embeddings in Sequence to Sequence NMT model. or any other means to do machine translation with a very small data set. The language is low resource language
import torch from transformers import XLMRobertaModel, XLMRobertaTokenizer tokenizer = XLMRobertaTokenizer.from_pretrained('xlm-roberta-large') model = XLMRobertaModel.from_pretrained('xlm-roberta-large') def get_ids(tokens, tokenizer, max_seq_length): token_ids = tokenizer.convert_tokens_to_ids(tokens) input_ids = token_ids +  * (max_seq_length - len(token_ids)) return input_ids s = "test sentence" stokens = tokenizer.tokenize(s) print(stokens) stokens = ["[CLS]"] + stokens + ["[SEP]"] input_ids = get_ids(stokens, tokenizer, 15) print(tokenizer.convert_tokens_to_ids(['test'])) print(tokenizer.convert_tokens_to_ids(['▁test'])) print(tokenizer.convert_ids_to_tokens()) print(tokenizer.convert_ids_to_tokens()) tokens_tensor = torch.tensor([input_ids]) print(input_ids) print(tokens_tensor) ```