I'm trying to write a program that uses Roberta to calculate word embeddings:

from transformers import RobertaModel, RobertaTokenizer
import torch

model = RobertaModel.from_pretrained('roberta-base')
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
caption = "this bird is yellow has red wings"

encoded_caption = tokenizer(caption, return_tensors='pt')
input_ids = encoded_caption['input_ids']

outputs = model(input_ids)
word_embeddings = outputs.last_hidden_state

I extract the last hidden state after forwarding the input_ids to the RobertaModel class to calculate word embeddings, I don't know if this is the correct way to do this, can anyone help me confirm this ? Thanks

I'm trying to write a program that using Roberta to calculate word embeddings:

from transformers import RobertaModel, RobertaTokenizer
import torch

model = RobertaModel.from_pretrained('roberta-base')
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
caption = "this bird is yellow has red wings"

encoded_caption = tokenizer(caption, return_tensors='pt')
input_ids = encoded_caption['input_ids']

outputs = model(input_ids)
word_embeddings = outputs.last_hidden_state

I extract the last hidden state after forwarding the input_ids to the RobertaModel class to calculate word embeddings, I don't know if this is the correct way to do this, can anyone help me confirm this ? Thanks