# Tag Info

Accepted

### What is the use of torch.no_grad in pytorch?

The wrapper with torch.no_grad() temporarily sets all of the requires_grad flags to false. An example is from the official ...
• 993
Accepted

### What loss function to use for imbalanced classes (using PyTorch)?

What kind of loss function would I use here? Cross-entropy is the go-to loss function for classification tasks, either balanced or imbalanced. It is the first choice when no preference is built from ...
• 8,479
Accepted

### Strange behavior with Adam optimizer when training for too long

This small instability at the end of convergence is a feature of Adam (and RMSProp) due to how it estimates mean gradient magnitudes over recent steps and divides by them. One thing Adam does is ...
• 27.2k

### How to get sentence embedding using BERT?

There is actually an academic paper for doing so. It is called S-BERT or Sentence-BERT. They also have a github repo which is easy to work with.

Assuming both of x_data and labels are lists or numpy arrays, ...
• 250
Accepted

### model.cuda() in pytorch

model.cuda() by default will send your model to the "current device", which can be set with torch.cuda.set_device(device). An ...
• 15k

### What is the use of torch.no_grad in pytorch?

Torch.no_grad() deactivates autograd engine. Eventually it will reduce the memory usage and speed up computations. Use of ...
• 161
Accepted

### How does the forward method get called in this pyTorch conv net?

If you look at the Module implementation of pyTorch, you'll see that forward is a method called in the special method __call__ : ...
• 1,071
Accepted

### Determining size of FC layer after Conv layer in PyTorch

Hello and welcome to Stack Exchange! The answer to your question is quite simple: you did not use the correct formula. The formula you used is (assuming we are working with square inputs)  W'=\...
• 226

### What is the use of torch.no_grad in pytorch?

with torch.no_grad() will make all the operations in the block have no gradients. In pytorch, you can't do inplacement changing of w1 and w2, which are two ...
• 101
Accepted

### PyTorch vs. Tensorflow Fold

There are a couple of good threads on Reddit right now (here and here). I haven't used either of these frameworks, but from reading around and talking to users I gather that support for dynamic ...
Accepted

### What is the difference between Pytorch's DataParallel and DistributedDataParallel?

As the Distributed GPUs functionality is only a couple of days old [in the v2.0 release version of Pytorch], there is still no documentation regarding that. So, I had to go through the source code's ...
• 7,996

### How to get sentence embedding using BERT?

Which vector represents the sentence embedding here? Is it hidden_reps or cls_head? If we look in the ...
• 2,261
Accepted

### An Artificial Neural Network (ANN) with an arbitrary number of inputs and outputs

The answer may depend on the significance of the length of the input vector or how it originates. However, the simplest solution is usually to know the largest size input and use that as number of ...
• 226

I think the standard way is to create a Dataset class object from the arrays and pass the Dataset object to the ...
• 171
Accepted

### Implementation of BERT using Tensorflow vs PyTorch

There are not only 2, but many implementations of BERT. Most are basically equivalent. The implementations that you mentioned are: The original code by Google, in Tensorflow. https://github.com/...
• 15k
Accepted

### Does torch.cat work with backpropagation?

Yes, torch.cat works with backward operation.
• 1,108

### How to install pytorch in windows?

As of August 14, 2017, you can install Pytorch from peterjc123's fork as follows. Currently, python 3.5 and 3.6 are supported. ...
• 375
Accepted

### How to convert my tensorflow model to pytorch model?

You can build the same model in pytorch. Then extract weights from tensorflow and assign them manually to each layer in pytorch. Depending on the amount of layers it could be time consuming. Building ...
• 1,169

### Autoencoder: using cosine distance as loss function

Hey so the Keras implementation of Cosine Similarity is called as Cosine Proximity. It just has one small change, that being ...

### What is the difference between Pytorch's DataParallel and DistributedDataParallel?

DataParallel is easier to debug, because your training script is contained in one process. DataParallel may also cause poor GPU-...
• 51

### Differences between gradient calculated by different reduction methods in PyTorch

Let's start by just recalling what each of these means. Reduction 'none' means compute batch_size gradient updates independently for the loss with respect to each ...
• 161
Accepted

### Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

After a Googling around, I think this tutorial may suit your needs. However, it seems you have a misconception about the Transformer decoder: in training mode there is no iteration at all. While LSTM-...
• 15k

### Is time series forecasting possible with a transformer?

Transformers can be used for time series forecasting. See the following articles: Adversarial Sparse Transformer for Time Series Forecasting, by Sifan Wu et al. Deep Transformer Models for Time ...

### An Artificial Neural Network (ANN) with an arbitrary number of inputs and outputs

I think you might have misunderstood the fixed number of inputs for the RNN. This is the number of inputs per timestep. All your examples have a fixed number of inputs per timestep: 1! You feed them ...
• 141

### How/What to initialize the hidden states in RNN sequence-to-sequence models?

It is important to clear up the difference between hidden state initialization and weight initialization. Glotrot (Xavier), Kaiming etc. are all initialization methods for the weights of neural ...
• 85

### Any pytorch tools to monitor neural network's training?

I am using tensorboardX. It supports most of the features of TensorBoard. I am using the Scalar, Images, Distributions, Histograms and Text. Haven't tried the rest like audio and graph. But the repo ...