Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Validation error is slightly lower than training error, but only for some initial conditions

Fair warning, I'm new to this field, so my process may be odd. Any advice is appreciated. I am currently training a model to reproduce some DFT (density functional theory) data. I have been doing ...
0
votes
0answers
5 views

How does embedding layer works in pytorch with neural machine translation?

as i mentioned on title, How does pytorch embedding layer works in machine translation task ? As i know that we can use CBOW or Skip-gram to create pretrained embedding vectors for our translation ...
2
votes
0answers
17 views

SpaCy vs AllenNLP?

I have used a little of both spaCy and allenNLP in my NLP projects. I like them both as they work very well with PyTorch (my DL framework choice!). But, I still cannot decide which one to master in a ...
0
votes
1answer
11 views

Two different pytorch networks, combined loss, back propagation and optimizer step

So here is my network architecture as shown in below image. I’ve two separate networks and loss_1 and loss_2 are coming from two ...
2
votes
1answer
15 views

Why use different variations of Softmax in training and validation for neural networks with Pytorch?

Specifically, I'm working on a modeling project, and I see someone else's code that looks like ...
0
votes
1answer
15 views

Multiply weights after using dropout in training - PyTorch

I have a Pytorch regression model as follows: ...
0
votes
0answers
8 views

Preprocessing data in image segmentation problem

I am implementing a research paper on image segmentation. Following are the image segmentation steps which are to be done before training its network- 1.Following image normalization is used- ...
5
votes
0answers
36 views

Differences between gradient calculated by different reduction methods in PyTorch

I'm playing with different reduction methods provided in built-in loss functions. In particular, I would like to compare the following. The averaged gradient by performing backward pass for each loss ...
0
votes
0answers
11 views

Multiple inputs to Multiple Neural network in parallel in Keras or Pytorch

I want to make a deep network as shown in the image. I want each 'network 1' to look at the specific part of the input and I don't want to divide my input beforehand in chunks. Is there any such ...
1
vote
1answer
11 views

Pytorch - Gradient distribution between functions

https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/neural_networks_tutorial.ipynb Hi I am trying to understand the NN with pytorch. I have doubts in gradient ...
0
votes
0answers
19 views

Replication of Andrew Ng's Sparse Autoencoder

for the past three days I have been trying to replicate the results presented in Andrew Ng's sparse autoencoding lecture (https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf) however I have ...
0
votes
1answer
16 views

Pytorch convolution input reshaping

I am new to CNNs, and I'm trying to follow along with a Pytorch DCGAN tutorial by reimplementing it in Keras. Clearly there are some differences in the frameworks, but in particular I am struggling to ...
0
votes
0answers
10 views

DataParallel and cuda()

Do we need to call cuda() for model and data if we use DataParallel? Say we have four GPUs, specifically there are three ...
0
votes
0answers
11 views

Out of memory when computing a Jacobian

I have this network: ...
1
vote
1answer
20 views

model.cuda() in pytorch

If I call model.cuda() in pytorch where model is a subclass of nn.Module, and say if I have four GPUs, how it will utilize the ...
0
votes
0answers
21 views

Transformer for neural machine translation: is it possible to predict each word in the target sentence in a single forward pass?

I want to replicate the Transformer from the paper Attention Is All You Need in PyTorch. My question is about the decoder branch of the Transformer. If I understand correctly, given a sentence in the ...
0
votes
2answers
35 views

When to use model.train() vs model.eval() in Pytoch?

I have a model that is used in a reinforcement learning algorithm for checkers, a la AlphaZero. Similar to that network, mine features batch normalization after each convolution layer. I am aware that ...
0
votes
1answer
46 views

r“”“What does it mean?”“” [closed]

I see r""" followed by a comment in quite a few of the source codes for PyTorch and the one I am looking at is here. What is the ...
2
votes
0answers
49 views

How to add a CNN layer on top of BERT?

I am just playing with bert (Bidirectional Encoder Representation from Transformer) Research Paper Suppose I want to add any other model or layers like Convolutional Neural Network layers (CNN), Non ...
0
votes
0answers
27 views

Text classifiaction for large datasets using Transfer learning

I am trying to do text classification on a very large set of documents using the pretrained GPT model. The problem is GPT takes max sequence length $\le$ 1024. I can't truncate the data as I need to ...
0
votes
0answers
13 views

Finetuning BERT

Referring to the PyTorch port by huggingface of the native BERT library, I want to fine-tune the generated model on my personal dataset containing raw text. Could you please point out how this can be ...
0
votes
0answers
17 views

Image is in JPEG but Torchvision shows image extension is unsupported

I just resized the image dataset with Pillow and exported to JPEG mydata = dsets.ImageFolder(data_path_here, transform=transform) Image is exported in JPG format ...
1
vote
0answers
17 views

A model that only works by setting all initial weights to zero

In this model from MusicNet, they set the initial weights of their neural network to all zeros. ...
1
vote
1answer
22 views

What are the input and output channels of a convolution in PyTorch?

From the documentation of Pytorch for Convolution, I saw the function torch.nn.Conv1d requires users to pass the parameters "in_channels" and "out_channels". I know they refer to input channels and ...
1
vote
0answers
30 views

Derivative of the Jacobian

I want to take the derivate of the jacobian using pytorch, but it seems like I am doing the wrong thing. Here is part of my code: ...
1
vote
0answers
34 views

Pytorch: How to implement nested transformers: a character-level transformer for words and a word-level transformer for sentences?

I have a model in mind, but I'm having a hard time figuring out how to actually code it in Pytorch, especially when it comes to training the model (e.g. how to define mini-batches, etc.). First of all ...
0
votes
0answers
26 views

Fast.ai Learner.validate result dependent on used sampler or num_workers in DataLoader

I am trying to understand why the result of Learner.validate function in fastai library depends heavily on the sampler used in <...
0
votes
0answers
20 views

How to create custom dataloader in pytorch for an input shape of (242,242,12), where 12 is the number of slices of a 3d MRI Nifti image

I am trying a 2D convolution network and I have had difficulty in creating the custom dataloader for my input images which are slices(12) of MRI image of size (242,242). I have created the network ...
1
vote
0answers
15 views

Write way to add samples to torch TabularDataset

I have a TabularDataset and i would like to add some examples to the dataset. ...
0
votes
0answers
14 views

OSError in Kaggle using [Fastai]

I am trying to use fastai (v 1.0.52) in Kaggle and have been getting the following error very time call the tabular_learner or conv_learner This error does not occurs when using ...
1
vote
1answer
62 views

Am I using optim.SGD incorrectly in pytorch?

I am doing reinforcement learning in checkers. After each game the network beats itself, I calculate the loss of every individual position in the game, call backward(), and step(). I am beginning to ...
1
vote
1answer
22 views

Saving the hidden layers of a trained network

I have used a pre-trained VGG-19 model to build an image classifier as part of a MOOC. I have implemented a classifier like this: ...
2
votes
1answer
27 views

How to optimize the lambdas of a hybrid loss in a deep learning model

I am using a generative adversarial deep learning model (GAN) with a hybrid loss represented by a linear combination of four losses with three $\lambda$'s, something like: $total\_loss = loss_1 + \...
0
votes
1answer
45 views

How to combine different kernels for Gaussian process in GPyTorch?

I am trying to learn gaussian process by using GPyTorch to fit a Gaussian Process Regression model. However, I can't figure out ...
0
votes
0answers
102 views

RL ppo alrorithm: understanding value loss and entropy plot

I'm implementing a computer vision program using PPO alrorithm mostly based on this work Both the critic loss and the actor loss decrease in the first serveal hundred episodes and keep near 0 later(...
0
votes
0answers
22 views

Pytorch's pack_padded_sequence in Tensorflow?

If we do not use pack_padded_sequence of Pytorch, what will happen to the eval result? How to implement Pytorch's ...
1
vote
1answer
33 views

conv2d function in pytorch

I'm trying to use the function torch.conv2d from Pytorch but can't get a result I understand... Here is a simple example where the kernel (filt) is the same size ...
1
vote
1answer
70 views

Simple linear regression in PyTorch

I am performing simple linear regression using PyTorch but my model is not able to properly fit over the training data. please look at the code to find the mistake. Dataset is here ...
0
votes
0answers
27 views

Which is the “most properly working” Bert-Ner repository

I am trying to find a repository in Github to get a Pytorch-reimplementation of the Bert model for NER task. So far, I found the following repos: https://github.com/kamalkraj/BERT-NER https://github....
1
vote
2answers
196 views

GAN - am I seeing mode collapse? Common fixes not working

I have a 2 part question. Context I am learning about GANs and writing my own starting from the very simplest example of adversarial learning (1-parameter node), then implementing a very simple 1-...
0
votes
0answers
30 views

PyTorch does not converge when approximating square function with linear model

I'm trying to learn some PyTorch and am referencing this discussion here The author provides a minimum working piece of code that illustrates how you can use PyTorch to solve for an unknown linear ...
0
votes
1answer
35 views

How does DQN solve Open AI Cartpole - v0?

Context I am confused about how a DQN is supposed to solve the cart pole problem since the rewards are so dense. I have been using pytorch example. I am aware of some solutions, but I have issue with ...
0
votes
0answers
36 views

Label embedding in Auxiliary Classifier GANs

In Auxiliary Classifier GAN the generator takes two inputs, 1. one hot encoding of the labels, and 2. noise vector. But in the implementation of the GAN (e.g.:) some embedding is used, I think it is ...
0
votes
0answers
43 views

Places365 for pytorch

I'm trying to use Places365 (the Vgg implementation) in PyTorch. I downloaded the model and the weights from the repo. The Vgg16 version of Places365 found in the official Github repo contains a ...
0
votes
0answers
31 views

Finetuning pretrained inception_v3 in pytorch

I'm following this tutorial but I'm having some trouble with inception. Every architecture works successfully, but when I run the tutorial code for inception, I get the following error: ...
0
votes
1answer
94 views

classification problem in pytorch with loss function CrossEntropyLoss returns negative output in prediction

I am trying to train and predict SVHN dataset (VGG architecture). I get very high validate/test accuracy by just getting the largest output class. However, the output weights are of large positive and ...
0
votes
0answers
9 views

Neural network that identify if tagging looks like a real tag

i want to build rnn that say how likely a tag (my other lstm) is a good fit for some sentences Which model should i use? I have data set of songs and matching chords... The lstm is input is sentence ...
0
votes
0answers
17 views

How to retain dependency between variables in PyTorch?

I am modeling k-dimensional positions over time t = 0...T using a set of initial positions Z0 with requires_grad=True and storing the results in Z with requires_grad=False for the remaining T-1 time ...