Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

0
votes
1answer
4 views

classification problem in pytorch with loss function CrossEntropyLoss returns negative output in prediction

I am trying to train and predict SVHN dataset (VGG architecture). I get very high validate/test accuracy by just getting the largest output class. However, the output weights are of large positive and ...
0
votes
0answers
7 views

Neural network that identify if tagging looks like a real tag

i want to build rnn that say how likely a tag (my other lstm) is a good fit for some sentences Which model should i use? I have data set of songs and matching chords... The lstm is input is sentence ...
0
votes
0answers
10 views

How to retain dependency between variables in PyTorch?

I am modeling k-dimensional positions over time t = 0...T using a set of initial positions Z0 with requires_grad=True and storing the results in Z with requires_grad=False for the remaining T-1 time ...
0
votes
0answers
9 views

Training error in RNN overshoots after considerable amount of iterations

I have trained a RNN model with pytorch. The training error overshoots after around 7.5k iterations. I have used gradient clipping and MultistepLR to decay the learning rate. Training Error on log ...
2
votes
1answer
38 views

Square Root Regularization and High Loss

I am testing out square root regularization (explained ahead) in a pytorch implementation of a neural network. Square root regularization, henceforth l1/2, is just like l2 regularization, but instead ...
1
vote
1answer
27 views

Pytorch : Loss function for binary classification

Fairly newbie to Pytorch & neural nets world.Below is a code snippet from a binary classification being done using a simple 3 layer network : ...
0
votes
0answers
21 views

Cannot Obtain Similar DL Prediction Result in Pytorch C++ API Compared to Python

I have trained a deep learning model using unet architecture in order to segment the nuclei in python and pytorch. I would like to load this pretrained model and make prediction in C++. For this ...
0
votes
0answers
15 views

Implementation of actor-critic model for MountainCar

I'm trying to build a model for the Mountain Car game, following this Actor-Critic code: https://github.com/nikhilbarhate99/Actor-Critic (However, in this case, it's discrete action space, while it's ...
2
votes
0answers
41 views

LSTM not converging

I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
5
votes
1answer
103 views

What loss function to use for imbalanced classes (using PyTorch)?

I have a dataset with 3 classes with the following items: Class 1: 900 elements Class 2: 15000 elements Class 3: 800 elements I need to predict class 1 and class 3, which signal important ...
0
votes
0answers
12 views

Speeding up actor critic training

I'm simulating a very simple system, recommendation system, and I am running an actor-critic model to predict what item I should recommend next. The agent is learning and is doing just fine. However, ...
2
votes
0answers
23 views

Moving to pytorch from tensorflow: practical considerations regarding inputs

As TF 2.0 looms and with it the certainty of having to rewrite or throw away most of my scripts, I am considering switching to pytorch. I initially liked TF for its low-level API — I think it is ...
1
vote
0answers
23 views

PyTorch does not seem to be optimizing correctly

I am trying to minimize the following function: $$f(\theta_1, \dots, \theta_n) = \frac{1}{s}\sum_{j =1}^s\left(\sum_{i=1}^n \sin(t_j + \theta_i)\right)^2$$ with respect to $(\theta_1, \dots, \...
1
vote
0answers
20 views

Reconstructing input image from layers of a CNN

I've been trying to implement neural style transfer as described in this paper here According to the paper, we can visualise the information at different processing stages in the CNN by ...
1
vote
0answers
16 views

What is the input for Conv2d for text?

I have an embedding layer that returns a tensor of type: batch x max_len x embeddings_dim (8 x 200 x 300) I want to input this to a Conv2d layer with kernel 3x3. I tried ...
1
vote
0answers
14 views

Weight update to fully convolutional network when supervision is only for a patch

I had a fundamental question, that is independent of any DL framework. In a fully convolutional network, if we forward an image of size 1000 x 1000, but only provide supervision signal for a 100 x 100 ...
3
votes
1answer
31 views

How to choose the number of output channels in a convolutional layer?

I'm following a pytorch tutorial where for a tensor of shape [8,3,32,32], where 8 is the batch size, 3 the number of channels and 32 x 32, the pixel size, they define the first convolutional layer as ...
2
votes
1answer
37 views

MSE loss different in Keras and PyToch

My problem is that in PyTorch I cannot reproduce the MSE loss that I have achieved in Keras. I have trained the following model in Keras: ...
1
vote
1answer
25 views

Pytorch dynamic forward pass

Does there exist a fast and convenient way for handling such a problem: ...
0
votes
1answer
44 views

Can pytorch / keras support dataloader object of Image and Text?

Is there a way of creating a dataloader object, or the equivalent in Keras, where every observation is an image AND some text? I could create two models to do classification but I want to see if I can ...
2
votes
1answer
34 views

Different learning rate for each of the layers?

I noticed that some popular deep learning frameworks like Keras or Pytorch allow you to set different learning rate for each layer. What are the benefits of that approach?
2
votes
0answers
25 views

What’s more appropriate for outlier detection? Classification or Regression?

I'm using PyTorch and i’m working on a data set where I only care about “outliers” from the norm. The data is comprised of long time series with gaussian distribution, while once in a while there is a ...
1
vote
2answers
122 views

Loading own train data and labels in dataloader using pytorch?

I have x_data and labels separately. How can I combine and load them in the model using torch.utils.data.DataLoader? I have a dataset that I created and the ...
2
votes
0answers
97 views

Pytorch: How to create an update rule the doesn't come from derivatives?

I want to implement the following algorithm, taken from this book, section 13.6: Here, the neural networks' outputs are $V(S, w)$ and $\pi(A|S,\theta)$, parameterized by $w$ and $\theta$ respectively....
1
vote
1answer
132 views

gpu pytorch code way slower than cpu code?

I have the following pytorch code in a jupyter notebook: import torch t_cpu = torch.rand(500,500,500) %timeit t_cpu @ t_cpu Which outputs: ...
1
vote
1answer
19 views

Doubt regarding the number of weights in 2 layer neural network

Considering a hypothetical scenario , where we have 10 input layers, and 5 output layers. How many weights are there in the neural network? If this is implemented in pytorch, the answer will be 50. ...
1
vote
0answers
15 views

Word2vec compact models

Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
3
votes
0answers
24 views

Unsure of how to implement an equation in PyTorch

I am trying to implement the SummaRuNNer architecture ( Nallapati et al). The equation I am stuck at in question is: $$d = tanh(W_{d}\frac{1}{N_{d}}\sum_{j=1}^{N^{d}}[h^{f}_{j},h^{b}_{j}] + b)$$ ...
1
vote
1answer
10 views

Recognizing circled numbers on a piece of paper

I've built a handful of CNN using tensorflow, keras, pytorch for recognizing text/number/objects in an image. What I'm trying to figure out how to do now is how to recognize numbers on a piece of ...
2
votes
0answers
62 views

Why is PyTorch's DataLoader not deterministic?

I've set the seeds like this (hoping to cover all bases): ...
0
votes
0answers
61 views

Replicating RNN within PyTorch

I tried to create a manual RNN and followed the official PyTorch example, which tries to classify a name to a language. I should note that it does indeed work. I'm not using the final logsoftmax, ...
2
votes
0answers
80 views

static graphs v.s. dynamic graphs

In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
1
vote
0answers
11 views

Unusual Memory trends while using Densenet

I am using Densenet to train my model but I found some unusual trends of memory usage while training the network, it follows the normal curve. I am unable to understand this can somebody help me ...
0
votes
0answers
24 views

Bugs in Pytorch replication of a simple LSTM model built with Keras

I am new to Pytorch. I am trying to replicate a simple Keras LSTM model in Pytorch. Two model takes in the exact same data but the Pytorch implementation produces a significantly worse result. In my ...
0
votes
2answers
332 views

In Pytorch, if I have a 2D tensor, how to iterate over this tensor to get every value changed

I have a 2d Tensor, whose size is 1024x1024 and the values in the tensor is 0.3333, 0.6667, and 1.0000, so I would like to change all these values to 0,1,2. Could some one tell me how to iterate over ...
1
vote
0answers
20 views

How can one encode data of different dimensions into a tensor in Pytorch?

I'm relatively new to Deep Learning. In my Deep Learning class our input was always images or vectors of numbers which fit a nice rectangular format. My question is: if you have data that doesn't fit ...
0
votes
0answers
23 views

How to re-initialise batch sampling with pytorch dataloader?

How do I re-initialise the sampling of Dataloader (docs page here) in pytorch? What I mean is: If I iterate through half of my data using the pytorch dataloader, then break and start a new loop, will ...
0
votes
0answers
11 views

NLP - Researches about data oriented text generation

I am relatively new to NLP. I am looking for a guidance as to where I can look to find about current researches in data driven text-generation, or data driven chatbot. Has there been some researches ...
0
votes
1answer
77 views

Understanding a Neural Network with Keras (preferably), TensorFlow or PyTorch

I would like to try a technique I saw in an article a while ago where, in order to understand what each neuron is doing, you apply specific inputs to the net and see which one of them is most ...
0
votes
2answers
41 views

Gradient of NN output with respect to inputs

I've trained a neural net on a problem where multiple inputs can be mapped to the same output. I'd like to use this NN to go from an output to an input i.e. given an output vector $y$, I want to find ...
0
votes
1answer
52 views

Architecture for linear regression with variable input where each input is n-sized one-hot encoded

I am relatively new to deep learning (got some experience with CNNs in PyTorch), and I am not sure how to tackle the following idea. I want to parse a sentence, e.g. I like trees., one-hot encoded the ...
1
vote
0answers
68 views

get a can't set attribute while using GPU in google colab but not not while using CPU

Hi i was learning to create a classifier using pytorch in google colab that i learned in Udacity. here is the link so i was loading data in the dataloader and when i used cpu it loaded and displayed ...
3
votes
2answers
164 views

Tensorflow (or Keras) vs. Pytorch vs. some other ML library for implementing a CNN [closed]

I am looking into implementing a convolutional neural network for a research problem. I've heard of deep learning libraries like Pytorch and Tensorflow and was hoping to get some additional ...
1
vote
0answers
103 views

AttributeError: module 'torch.distributed' has no attribute 'init_process_group' [closed]

AttributeError: module 'torch.distributed' has no attribute 'init_process_group' Still getting this error after installing latest pytorch-nightly and other stuff, when I tried to run the imagenet ...
1
vote
0answers
22 views

LSTM Produces Random Predictions

I have trained an LSTM in PyTorch on financial data where a series of 14 values predicts the 15th. I split the data into Train, Test, and Validation sets. I trained the model until the loss ...
0
votes
1answer
43 views

Unsupervised learning from images [closed]

I want to design a model that can detect the different feature in the images, let's consider we have ~100000 images of cows. when I give this images to the model it has to identify different parts of ...
1
vote
1answer
91 views

How can I parallelize GloVe reverse lookups in PyTorch?

I feel like I'm missing something obvious here because I can't find any discussion of this. I want to do a lot of reverse lookups (nearest neighbor distance searches) on the GloVe embeddings for a ...
0
votes
1answer
36 views

Combining different features as input to Neural Network

I use two different sources of information as input to my neural model. The model takes a word as input and produces a 1/0 output. I represent each word by using its word embedding (1024 dimensional ...
0
votes
1answer
30 views

How can I create convolutions or linear layers that operate on vectors rather than scalars in pytorch?

Consider an nn.Linear(2,3) layer transform like the one below. It uses a 2x3 matrix of scalar weights to create a weighted sum for each scalar element in the ...
1
vote
0answers
16 views

Test Loss plateau fast in Convolutional Neural Net

I have a 10k dataset of 1 channel 100X100pixels images with 31 classes. I set up a CNN with 3 convolution layers each followed by a batchnorm and a 2d pooling. I tried out several combinations of ...