Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
7 votes
1 answer
9k views

Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.). For the purposes of this question, you may assume that I already have the input part ...
Pablo Messina's user avatar
27 votes
1 answer
10k views

PyTorch vs. Tensorflow Fold

Both PyTorch and Tensorflow Fold are deep learning frameworks meant to deal with situations where the input data has non-uniform length or dimensions (that is, situations where dynamic graphs are ...
noe's user avatar
  • 25.7k
16 votes
2 answers
15k views

Strange behavior with Adam optimizer when training for too long

I'm trying to train a single perceptron (1000 input units, 1 output, no hidden layers) on 64 randomly generated data points. I'm using Pytorch using the Adam optimizer: ...
Bai Li's user avatar
  • 263
7 votes
3 answers
7k views

Is time series forecasting possible with a transformer?

For my bachelor project I've been tasked with making a transformer that can forecast time series data, specifically powergrid data. I need to take a univariate time series of length N, that can then ...
Sebastian Eliassen's user avatar
5 votes
1 answer
19k views

How to convert my tensorflow model to pytorch model? [closed]

I performed transfer learning using ssd + mobilenet as my base model in tensorflow and freezed a new model. Now I want to convert that model into pytorch. Is there any way how I can achieve it? Any ...
Rachit Tayal's user avatar
4 votes
1 answer
2k views

How does the Transformer predict n steps into the future?

I have barely been able to find an implementation of the Transformer (that is not bloated nor confusing), and the one that I've used as reference was the PyTorch implementation. However, the Pytorch ...
skevelis's user avatar
2 votes
2 answers
215 views

What is the ELMO approach to learn contextual embedding?

BERT, GPT, and ELMo used the contextual embedding. but, their approach of learning contextual embedding is different. so, what is the ELMo approach to learn contextual embedding?
tovijayak's user avatar
2 votes
1 answer
2k views

Tutorial for restricted Boltzmann machine using PyTorch or Tensorflow?

I am trying to find a tutorial on training Restricted Boltzmann machines on some dataset (e.g. MNIST), using either PyTorch or Tensorflow. The few I found are outdated. Can you recommend any?
a06e's user avatar
  • 129
2 votes
1 answer
5k views

How does BERT and GPT-2 encoding deal with token such as <|startoftext|>, <s>

As I understand, GPT-2 and BERT are using Byte-Pair Encoding which is a subword encoding. Since lots of start/end token is used such as <|startoftext|> and , as I image the encoder should encode ...
Kevin Ling's user avatar
1 vote
0 answers
128 views

Which One is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?

Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch. ...
bbasaran's user avatar
  • 171
1 vote
1 answer
1k views

Requirements for variable length output in transformer

I have been working on modifying the transformer from the article The Annotated Transformer. One of the features I would like to include is the ability to pass a sequence of fixed length, and receive ...
try_hard's user avatar
1 vote
2 answers
1k views

RNN with PyTorch - I don't understand the initial parameters

I would like to understand the pyTorch RNN module in detail. There I created a very simple and basic example: ...
Thomas K's user avatar
1 vote
1 answer
300 views

How to handle OOV in non-contextual embedding (word2vec, Glove, FastText)?

how non-contextual embedding (Word2Vec, Glove, FastText) handle OOV (incase if given word is not available in vocabulary)
tovijayak's user avatar
1 vote
1 answer
56 views

"model.to('cuda:6')" becomes (nvidia-smi) GPU 4, same with any other "cuda:MY_GPU", only "cuda:0" becomes GPU 0. How do I get rid of this mapping?

Strange mapping: example In the following example, the first column is chosen in the code, second column is the one that does the work instead: 0:0 1234 MiB 1:2 1234 MiB 2:7 1234 MiB 3:5 2341 MiB 4:1 ...
questionto42's user avatar
1 vote
1 answer
2k views

Use embeddings to find similarity between documents

I need to find cosine similarity between two text documents. I need embeddings that reflect order of the word sequence, so I don't plan to use document vectors built with bag of words or TF/IDF. ...
dokondr's user avatar
  • 295
0 votes
1 answer
108 views

Transformers Trainer: "RuntimeError: module must have its parameters ... on device cuda:6 (device_ids[0]) but found one of them on device: cuda:0"

I ask this since I could not fix it with the help of: Stack Overflow RuntimeError: module must have its parameters and buffers on device cuda:1 (device_ids[0]) but found one of them on device: cuda:2 ...
questionto42's user avatar
0 votes
1 answer
1k views

LSTM Multi-class classification for large number of classes

I want to build a model that classifies 473 classes -product categories-, but I'm facing a problem with loss not decreasing. Data I have almost 3,000 data points for each class -473 classes- (data ...
Khaled's user avatar
  • 103
0 votes
1 answer
968 views

pytorch dataloader tensor modification

T=tensor([101,123,414,463][234,903,313,341]...) train=TensorDataset(T) train_dataloader=Dataloader(train) Now I would like to update tensor ...
SS Varshini's user avatar