Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

2
votes
1answer
17 views

Pytorch: How to create an update rule the doesn't come from derivatives?

I want to implement the following algorithm, taken from this book, section 13.6: Here, the neural networks' outputs are $V(S, w)$ and $\pi(A|S,\theta)$, parameterized by $w$ and $\theta$ respectively....
1
vote
1answer
13 views

gpu pytorch code way slower than cpu code?

I have the following pytorch code in a jupyter notebook: import torch t_cpu = torch.rand(500,500,500) %timeit t_cpu @ t_cpu Which outputs: ...
1
vote
1answer
13 views

Doubt regarding the number of weights in 2 layer neural network

Considering a hypothetical scenario , where we have 10 input layers, and 5 output layers. How many weights are there in the neural network? If this is implemented in pytorch, the answer will be 50. ...
1
vote
0answers
13 views

Word2vec compact models

Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
3
votes
0answers
19 views

Unsure of how to implement an equation in PyTorch

I am trying to implement the SummaRuNNer architecture ( Nallapati et al). The equation I am stuck at in question is: $$d = tanh(W_{d}\frac{1}{N_{d}}\sum_{j=1}^{N^{d}}[h^{f}_{j},h^{b}_{j}] + b)$$ ...
1
vote
1answer
10 views

Recognizing circled numbers on a piece of paper

I've built a handful of CNN using tensorflow, keras, pytorch for recognizing text/number/objects in an image. What I'm trying to figure out how to do now is how to recognize numbers on a piece of ...
2
votes
0answers
17 views

Why is PyTorch's DataLoader not deterministic?

I've set the seeds like this (hoping to cover all bases): ...
0
votes
0answers
19 views

Replicating RNN within PyTorch

I tried to create a manual RNN and followed the official PyTorch example, which tries to classify a name to a language. I should note that it does indeed work. I'm not using the final logsoftmax, ...
2
votes
0answers
13 views

static graphs v.s. dynamic graphs

In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
1
vote
0answers
9 views

Unusual Memory trends while using Densenet

I am using Densenet to train my model but I found some unusual trends of memory usage while training the network, it follows the normal curve. I am unable to understand this can somebody help me ...
0
votes
0answers
12 views

Bugs in Pytorch replication of a simple LSTM model built with Keras

I am new to Pytorch. I am trying to replicate a simple Keras LSTM model in Pytorch. Two model takes in the exact same data but the Pytorch implementation produces a significantly worse result. In my ...
0
votes
2answers
29 views

In Pytorch, if I have a 2D tensor, how to iterate over this tensor to get every value changed

I have a 2d Tensor, whose size is 1024x1024 and the values in the tensor is 0.3333, 0.6667, and 1.0000, so I would like to change all these values to 0,1,2. Could some one tell me how to iterate over ...
1
vote
0answers
15 views

How can one encode data of different dimensions into a tensor in Pytorch?

I'm relatively new to Deep Learning. In my Deep Learning class our input was always images or vectors of numbers which fit a nice rectangular format. My question is: if you have data that doesn't fit ...
0
votes
0answers
13 views

How to re-initialise batch sampling with pytorch dataloader?

How do I re-initialise the sampling of Dataloader (docs page here) in pytorch? What I mean is: If I iterate through half of my data using the pytorch dataloader, then break and start a new loop, will ...
0
votes
0answers
10 views

NLP - Researches about data oriented text generation

I am relatively new to NLP. I am looking for a guidance as to where I can look to find about current researches in data driven text-generation, or data driven chatbot. Has there been some researches ...
0
votes
1answer
67 views

Understanding a Neural Network with Keras (preferably), TensorFlow or PyTorch

I would like to try a technique I saw in an article a while ago where, in order to understand what each neuron is doing, you apply specific inputs to the net and see which one of them is most ...
0
votes
2answers
34 views

Gradient of NN output with respect to inputs

I've trained a neural net on a problem where multiple inputs can be mapped to the same output. I'd like to use this NN to go from an output to an input i.e. given an output vector $y$, I want to find ...
0
votes
1answer
47 views

Architecture for linear regression with variable input where each input is n-sized one-hot encoded

I am relatively new to deep learning (got some experience with CNNs in PyTorch), and I am not sure how to tackle the following idea. I want to parse a sentence, e.g. I like trees., one-hot encoded the ...
1
vote
0answers
24 views

get a can't set attribute while using GPU in google colab but not not while using CPU

Hi i was learning to create a classifier using pytorch in google colab that i learned in Udacity. here is the link so i was loading data in the dataloader and when i used cpu it loaded and displayed ...
0
votes
0answers
15 views

CNNs for time series data - concept of channels in Pytorch

This is somewhat related to another post I made Wrangling data for CNN. When inputting an image into a CNN, we have to define a parameter called in_channels in ...
3
votes
2answers
118 views

Tensorflow (or Keras) vs. Pytorch vs. some other ML library for implementing a CNN [closed]

I am looking into implementing a convolutional neural network for a research problem. I've heard of deep learning libraries like Pytorch and Tensorflow and was hoping to get some additional ...
0
votes
0answers
63 views

Comparison of Pytorch high level libraries

I have used keras for a few months and it provided me with some very useful high-level api such as it's callbacks. I am now interested in starting using pytorch and i'm interested to be assisted with ...
0
votes
0answers
22 views

Machine Learning VGG CNN M 1024 Network

can Someone explain me what is difference between VGG-16 and VGG-CNN-M-1024. I cant find any info. I am going to implement this model in pytorch (It's strange that they dont have any pretrained model ...
1
vote
0answers
65 views

AttributeError: module 'torch.distributed' has no attribute 'init_process_group' [closed]

AttributeError: module 'torch.distributed' has no attribute 'init_process_group' Still getting this error after installing latest pytorch-nightly and other stuff, when I tried to run the imagenet ...
1
vote
0answers
21 views

LSTM Produces Random Predictions

I have trained an LSTM in PyTorch on financial data where a series of 14 values predicts the 15th. I split the data into Train, Test, and Validation sets. I trained the model until the loss ...
0
votes
1answer
36 views

Unsupervised learning from images [closed]

I want to design a model that can detect the different feature in the images, let's consider we have ~100000 images of cows. when I give this images to the model it has to identify different parts of ...
0
votes
0answers
17 views

PyTorch BCELoss goes up with each epoch

the algorithm I am working on uses a Sigmoid output layer and a BCE Loss, but with each epoch the loss seems to be going up. Is the output of the BCELoss the prediction percentage? Otherwise, I'm ...
1
vote
1answer
34 views

How can I parallelize GloVe reverse lookups in PyTorch?

I feel like I'm missing something obvious here because I can't find any discussion of this. I want to do a lot of reverse lookups (nearest neighbor distance searches) on the GloVe embeddings for a ...
0
votes
1answer
31 views

Combining different features as input to Neural Network

I use two different sources of information as input to my neural model. The model takes a word as input and produces a 1/0 output. I represent each word by using its word embedding (1024 dimensional ...
0
votes
1answer
21 views

How can I create convolutions or linear layers that operate on vectors rather than scalars in pytorch?

Consider an nn.Linear(2,3) layer transform like the one below. It uses a 2x3 matrix of scalar weights to create a weighted sum for each scalar element in the ...
1
vote
0answers
9 views

Test Loss plateau fast in Convolutional Neural Net

I have a 10k dataset of 1 channel 100X100pixels images with 31 classes. I set up a CNN with 3 convolution layers each followed by a batchnorm and a 2d pooling. I tried out several combinations of ...
0
votes
1answer
26 views

Policy gradient - and auto-differentiation (Pytorch/Tensorflow)

In policy gradient, we have something like this: Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
0
votes
0answers
10 views

Theoretical underpinning behind Hardmax operator

In the tensor flow Github repository, in the file attentionwrapper.py, hardmax operator has been defined. On the docs, it has been mentioned tf.contrib.seq2seq.hardmax I want to know what's the ...
1
vote
2answers
161 views

How to force pytorch model to predict only positive values

I am doing a prediction on a data set where labels have positive values (time values). After training a simple Linear pytorch model I get negative values for time despite being 0 negative values in ...
0
votes
1answer
489 views

Determining size of FC layer after Conv layer in PyTorch

I am learning PyTorch and CNNs but am confused how the number of inputs to the first FC layer after a Conv2D layer is calculated. My network architecture is shown below, here is my reasoning using ...
0
votes
1answer
725 views

How to convert my tensorflow model to pytorch model? [closed]

I performed transfer learning using ssd + mobilenet as my base model in tensorflow and freezed a new model. Now I want to convert that model into pytorch. Is there any way how I can achieve it? Any ...
0
votes
0answers
80 views

How does the loss function for semantic segmentation networks (like FCN) work?

Just want to understand how the cost function works when performing semantic segmentation. I know that for simple classification networks, the output is a fully connected layer equal to the number of ...
1
vote
0answers
14 views

When training a neural network on a windows machine, what can task manager tell me about how it's going?

When I trained a simple convolutional network in pytorch, I could see in task manager that my cpu was hitting 100%, and I could see my ram spike up. When running with CUDA, I could see my GPU, go ...
1
vote
0answers
22 views

Deep RL: Visualizing/Analyzing the gradient

I am testing different RL methods, and I know e.g that policy gradient method is supposed to have a high variance gradient which can cause trouble. I want to run a few different Deep RL algorithms, ...
6
votes
1answer
109 views

Combining 2 Neural Networks

2 images as input, x1 and x2 and try to use convolution as a similarity measure. The idea is that the learned weights substitute more traditional measure of similarity (cross correlation, NN, ...). ...
0
votes
1answer
86 views

Loss function when the output is a single probability

I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1] ...
0
votes
0answers
82 views

Using Upsample instead of ConvTranspose2d causes cuda out of memory on gradient calculating step

Video card: gtx1070ti 8Gb, batchsize 64, input image size 128*128. I had such UNET with resnet152 as encoder which worked pretty fine: ...
2
votes
1answer
51 views

Getting rid of maxpooling layer causes running cuda out memory error pytorch

Video card: gtx1070ti 8Gb, batchsize 64. I had such UNET with resnet152 as encoder wich worket pretty fine: ...
2
votes
1answer
36 views

Gradient computation

I am beginner in data-science. I am trying to understand this PyTorch code for gradient computation using custom autograd function: ...
2
votes
0answers
151 views

Output size is too small for SpatialAveragePooling in Unet

Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
0
votes
0answers
65 views

How to compute individual loss w.r.t each input in mini-batch in PyTorch?

I am trying to create adversarial images using FGSM by Goodfellow. For this, I need to compute loss w.r.t each image in the mini-batch and not average of the individual losses. I can achieve this in ...
0
votes
0answers
48 views
1
vote
0answers
46 views

Time series pixel classification

Working on an classification problem with images at the pixel level using either keras(tf) or pytorch. The images are all 10,000 pixels wide and high. To elaborate there are two kinds of pixels in ...
2
votes
2answers
990 views

How to use Cross Entropy loss in pytorch for binary prediction?

In the pytorch docs, it says for cross entropy loss: input has to be a Tensor of size (minibatch, C) Does this mean that for binary (0,1) prediction, the input must be converted into an (N,2) ...
1
vote
1answer
70 views

Query on unstable loss curves for RNN

I’m currently building sequence models for forecasting, and have tried using RNNs, LSTMs, and GRUs. Something unusual I noticed was the highly unstable loss curves, where the loss sometimes goes back ...