Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

0
votes
0answers
9 views

Machine Learning VGG CNN M 1024 Network

can Someone explain me what is difference between VGG-16 and VGG-CNN-M-1024. I cant find any info. I am going to implement this model in pytorch (It's strange that they dont have any pretrained model ...
1
vote
0answers
16 views

AttributeError: module 'torch.distributed' has no attribute 'init_process_group' [on hold]

AttributeError: module 'torch.distributed' has no attribute 'init_process_group' Still getting this error after installing latest pytorch-nightly and other stuff, when I tried to run the imagenet ...
1
vote
0answers
12 views

LSTM Produces Random Predictions

I have trained an LSTM in PyTorch on financial data where a series of 14 values predicts the 15th. I split the data into Train, Test, and Validation sets. I trained the model until the loss ...
0
votes
1answer
30 views

Unsupervised learning from images

I want to design a model that can detect the different feature in the images, let's consider we have ~100000 images of cows. when I give this images to the model it has to identify different parts of ...
0
votes
0answers
7 views

PyTorch BCELoss goes up with each epoch

the algorithm I am working on uses a Sigmoid output layer and a BCE Loss, but with each epoch the loss seems to be going up. Is the output of the BCELoss the prediction percentage? Otherwise, I'm ...
1
vote
1answer
19 views

How can I parallelize GloVe reverse lookups in PyTorch?

I feel like I'm missing something obvious here because I can't find any discussion of this. I want to do a lot of reverse lookups (nearest neighbor distance searches) on the GloVe embeddings for a ...
0
votes
0answers
8 views

Implementation of Inception-Resnet V2 shapes does not match

I am trying to implement the Inception-Resnet V2. In the original paper the authors outlined the network as in the figure below: One can say that the output dimension of the earlier block is the ...
0
votes
1answer
15 views

Combining different features as input to Neural Network

I use two different sources of information as input to my neural model. The model takes a word as input and produces a 1/0 output. I represent each word by using its word embedding (1024 dimensional ...
0
votes
1answer
19 views

How can I create convolutions or linear layers that operate on vectors rather than scalars in pytorch?

Consider an nn.Linear(2,3) layer transform like the one below. It uses a 2x3 matrix of scalar weights to create a weighted sum for each scalar element in the ...
0
votes
0answers
30 views

Pytorch LSTM with dynamic batch size

I am using Pytorch to construct predictions for time series data. Each time step represents one day, and each observation on a given day has a fixed number of features. However, the number of ...
0
votes
0answers
31 views

GAN failing under WGANGP and LSGAN loss functions

I'm investigating the use of a Wasserstein GAN with gradient penalty in PyTorch. I'm heavily borrowing from Caogang's implementation, but am using the discriminator and generator losses used in this ...
0
votes
0answers
14 views

Loss function become worse after the data is shuffling

Is it possible that loss function become worse after the data is shuffling for every epoch (entire data)? In my case, I have around 85000 data with 7500 data attributes then I am using neural network ...
1
vote
0answers
5 views

Test Loss plateau fast in Convolutional Neural Net

I have a 10k dataset of 1 channel 100X100pixels images with 31 classes. I set up a CNN with 3 convolution layers each followed by a batchnorm and a 2d pooling. I tried out several combinations of ...
0
votes
0answers
12 views

Policy gradient - and auto-differentiation (Pytorch/Tensorflow)

In policy gradient, we have something like this: Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
0
votes
0answers
6 views

Theoretical underpinning behind Hardmax operator

In the tensor flow Github repository, in the file attentionwrapper.py, hardmax operator has been defined. On the docs, it has been mentioned tf.contrib.seq2seq.hardmax I want to know what's the ...
0
votes
2answers
42 views

How to force pytorch model to predict only positive values

I am doing a prediction on a data set where labels have positive values (time values). After training a simple Linear pytorch model I get negative values for time despite being 0 negative values in ...
0
votes
1answer
44 views

Determining size of FC layer after Conv layer in PyTorch

I am learning PyTorch and CNNs but am confused how the number of inputs to the first FC layer after a Conv2D layer is calculated. My network architecture is shown below, here is my reasoning using ...
0
votes
1answer
59 views

How to convert my tensorflow model to pytorch model? [closed]

I performed transfer learning using ssd + mobilenet as my base model in tensorflow and freezed a new model. Now I want to convert that model into pytorch. Is there any way how I can achieve it? Any ...
0
votes
0answers
47 views

How does the loss function for semantic segmentation networks (like FCN) work?

Just want to understand how the cost function works when performing semantic segmentation. I know that for simple classification networks, the output is a fully connected layer equal to the number of ...
1
vote
0answers
11 views

When training a neural network on a windows machine, what can task manager tell me about how it's going?

When I trained a simple convolutional network in pytorch, I could see in task manager that my cpu was hitting 100%, and I could see my ram spike up. When running with CUDA, I could see my GPU, go ...
1
vote
0answers
18 views

Deep RL: Visualizing/Analyzing the gradient

I am testing different RL methods, and I know e.g that policy gradient method is supposed to have a high variance gradient which can cause trouble. I want to run a few different Deep RL algorithms, ...
5
votes
0answers
64 views

Combining 2 Neural Networks

2 images as input, x1 and x2 and try to use convolution as a similarity measure. The idea is that the learned weights substitute more traditional measure of similarity (cross correlation, NN, ...). ...
0
votes
1answer
39 views

Loss function when the output is a single probability

I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1] ...
0
votes
0answers
42 views

Using Upsample instead of ConvTranspose2d causes cuda out of memory on gradient calculating step

Video card: gtx1070ti 8Gb, batchsize 64, input image size 128*128. I had such UNET with resnet152 as encoder which worked pretty fine: ...
2
votes
1answer
40 views

Getting rid of maxpooling layer causes running cuda out memory error pytorch

Video card: gtx1070ti 8Gb, batchsize 64. I had such UNET with resnet152 as encoder wich worket pretty fine: ...
2
votes
1answer
29 views

Gradient computation

I am beginner in data-science. I am trying to understand this PyTorch code for gradient computation using custom autograd function: ...
1
vote
0answers
37 views

Output size is too small for SpatialAveragePooling in Unet

Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
0
votes
0answers
46 views

How to compute individual loss w.r.t each input in mini-batch in PyTorch?

I am trying to create adversarial images using FGSM by Goodfellow. For this, I need to compute loss w.r.t each image in the mini-batch and not average of the individual losses. I can achieve this in ...
1
vote
0answers
42 views

Time series pixel classification

Working on an classification problem with images at the pixel level using either keras(tf) or pytorch. The images are all 10,000 pixels wide and high. To elaborate there are two kinds of pixels in ...
2
votes
2answers
478 views

How to use Cross Entropy loss in pytorch for binary prediction?

In the pytorch docs, it says for cross entropy loss: input has to be a Tensor of size (minibatch, C) Does this mean that for binary (0,1) prediction, the input must be converted into an (N,2) ...
1
vote
1answer
48 views

Query on unstable loss curves for RNN

I’m currently building sequence models for forecasting, and have tried using RNNs, LSTMs, and GRUs. Something unusual I noticed was the highly unstable loss curves, where the loss sometimes goes back ...
2
votes
1answer
90 views

Is GEMM used in Tensorflow, Theano, Pytorch

I know that Caffe uses GEneral Matrix to Matrix Multiplication (GEMM) which is part of Basic Linear Algebra Subprograms (BLAS) library for performing convolution operations. Where a convolution is ...
1
vote
1answer
290 views

Are view() in Pytorch and reshape() in Numpy similar?

Are view() in torch and reshape() in Numpy similar? view() is applied on torch tensors to ...
0
votes
1answer
55 views

Basic DNN with highly imbalanced dataset — network predicts same labels [closed]

I will try to explain my issue at a high level, and I hope I'd be able to get some better understanding of the ML behind it all. I am working with aggregated features extracted from audio files, so ...
1
vote
0answers
51 views

Pytorch - Torch.mm underflow error

When using torch.mm function I get an underflow error such that I get an output matrix of NaNs that is passed to subsequent layers of my model. How can I avoid this ...
1
vote
0answers
13 views

Convolutional Neural Network with feature inputs also directly connected to fullly connected layers

Imagine you are trying to classify movie reviews you have both the actual review and also some information about the movie. I pass the actual review through a few convolutional layers and some max ...
1
vote
0answers
33 views

Inseting pretrained network to pytorch

I am new to deep learning as well as pytorch. I have got the following code (siamese network) working with pytorch: https://github.com/KrisG04/Siamese-using-PyTorch/blob/master/Siamese-networks-...
1
vote
1answer
62 views

2D Pytorch tensor doesn't have independent random values

I've written some Python to create a pytorch tensor of random values, sampled from a Student's t distribution with 10 degrees of freedom: ...
0
votes
0answers
382 views

Enable Mini-batch Processing on PyTorch Word Embeddings

I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly. ...
3
votes
1answer
8k views

What is the use of torch.no_grad in pytorch?

I am new to pytorch and started with this github code. I do not understand the comment in line 60-61 in the code ...
0
votes
0answers
69 views

Pytorch: Combining Automatic and Manual Methods

I am testing a two-step architecture that is composed of a conventional first section that can be implemented with any standard deep learning architecture and a second section that must be coded ...
2
votes
2answers
324 views

NN embedding layer

Several neural network libraries such as tensorflow and pytorch offer an Embedding layer. Having implemented word2vec in the past, I understand the reasoning behind wanting a lower dimensional ...
1
vote
0answers
173 views

FastAI to PyTorch conversion

I have been taking the FastAI course. And if anyone else has taken this course they know that they use their own python library called fastai that is a wrapper for ...
0
votes
3answers
468 views

Sparse connections in feedforward network tensorflow or pytorch?

I want to create sparse feed-forward networks in Pytorch and Tensorflow, i.e., say each node is only connected to k number of neurons of the next layer where k is strictly less than the total number ...
2
votes
2answers
194 views

Help me choose a Data Science book in Python [closed]

I've been a Data Scientist for a few years now, but I've only recently started to do most of my work in Python (boy, do I miss ggplot2! But ...
0
votes
0answers
75 views

Variational Auto Encoder - Positive ELBO

I'm trying to run an example for a Recurrent Variational Auto Encoder using this piece of code (https://github.com/masonsun/deep_forecasting). Using the toy dataset provided, I come across a funny ...
0
votes
0answers
533 views

Size mismatch, m1: [1 x 3512320], m2: [2048 x 1000] at /opt/conda/conda-bld/pytorch_1513368888240/work/torch/lib/TH/generic/THTensorMath.c:1416

I am trying to do image segmentation. My image size is 1296x966 But ,I am getting size mismatch, m1: [1 x 3512320], m2: [2048 x 1000] at /opt/conda/conda-bld/pytorch_1513368888240/work/torch/lib/...
3
votes
2answers
575 views

Dropout Decreases Test and Train Accuracy in one layer LSTM in Pytorch

I have a one layer lstm with pytorch on Mnist data. I know that for one layer lstm dropout option for lstm in pytorch does not operate. So, I have added a drop out at the beginning of second layer ...
0
votes
0answers
29 views

Pytorch Capabilities

Is it possible to use the Levenshtein (edit) distance of two strings as the error function my model attempts to optimize? The inputs are images representing the letters in a text document. The output ...