Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

How many outputs for CNN when dealing with a multi-label classification problem with OneHot Encoded labels?

My labels are of type tensor [1 0 0 1] denoting a 4 label multiclass problem for any given 16x16 image. I'm using BCEWithLogitsLoss from ...
1
vote
1answer
22 views

How should I sample from a mixture distribution?

Let's say we have a mixture distribution, defined by density $f(x)= w_1 p_1(x) + w_2 p_2(x)$, where $w_i$ is a scalar weight. Furthermore, we have efficient methods to evaluate the pdf and cdf/icdf ...
1
vote
1answer
30 views

High validation loss, high validation accuracy

I'm just getting started into the field of deep learning, and I completed my first model training using PyTorch. I decided to use a pre-build model from torchvision, more specifically the mobilenet_v2 ...
0
votes
0answers
17 views

Bert question answering start_positions is larger than end_positions [closed]

I want to fine-tune Bert on question answering for a closed domain, so I started by discovering how it works first, i executed the code bellow but the result is no write, the start position is larger ...
0
votes
0answers
9 views

Unable to understand the data format while generating the data in parallel [closed]

I am trying to generate the data in parallel using this tutorial. I am using it for image classification problem and I have image data in jpeg format. I have a very basic doubt, in the below code, the ...
2
votes
1answer
52 views

Loss function for age classification

I am building a CNN model for age classification. Assuming age of a person is between 1-100, my last Linear Layer contains 100 output neuron. Now i want to find an appropriate loss function for this ...
0
votes
1answer
16 views

PyTorchs ConvTranspose2d padding parameter

Im confused about what PyTorchs padding parameter does when using torch.nn.ConvTranspose2d. The docs say that: "The padding argument effectively adds dilation * (kernel_size - 1) - padding ...
0
votes
1answer
26 views

NN Making Poor Averaging Fit whilst LGBM Regressor Fits Perfectly

I have a simple toy dataset for which the features have been encoded using a Encoder-Decoder NN. I am using the hidden feature vector from the Encoder as the X input for training a 1-step lookahead ...
0
votes
0answers
22 views

How to custom operations in the forward function in pytorch

I am doing an experiment on images of small size as follows: The input image has only one channel, then I apply 2 customed filters (denoted by A and B) to do the convolution with the image. The filter ...
0
votes
0answers
6 views

Is the first Linear layer in Pytorch considered as the input layer?

For example: nn.Linear(100,50) You have 100 input features and 50 outputs. Would this be considered as the input layer or already as a kind of second layer? The problem: I don't want to apply dropout ...
0
votes
1answer
17 views

Python (Pytorch) loss function syntax

I have seen many examples of this syntax that is being used for the loss function specifically: loss = nn.BCEWithLogitsLoss()(pred, y) Can anyone explain me what does the (pred, y) do exactly, that it ...
1
vote
0answers
28 views

LSTM model prediction scaling with loaded model

I am deploying a LSTM pytorch model for production and I have issue with scaling the LSTM output correctly. While the model was tested the output was scaled with label data: ...
1
vote
0answers
15 views

Why yolo4 pytorch re-training loss seems high as like first time training?

I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git. I generated checkpoints by giving ...
0
votes
1answer
12 views

How to automatically disable register_hook when training model is in eval() phase in PyTorch?

I require to update grads of an intermediate tensor variable using the register_hook method. Since the variable isn't a leaf-variable, I require to add the ...
0
votes
1answer
24 views

Why does the non autoregresive transfomer model in fairseq require the prev_output_tokens input?

fairseq includes an implementation of a non autoregressive transformer - which (as much as I understand) means that the whole output sequence is generated in a single forward run (in contrast to ...
0
votes
1answer
13 views

LSTM / GRU weights during test time

I am working on a historic time series dataset and using RNN, LSTM, GRU models, and I didn't find an answer if in test time, the h (or h, c) weights should be zeors for each batch? If the weights ...
0
votes
0answers
27 views

Which neural network library should be learned? [closed]

As I understand, Tensorflow+Keras, PyTorch, Theano, Kaffe are some of the libraries that can be used to develop NNs. I am curious about the following: Which one is most used and should be learned? ...
0
votes
0answers
16 views

Valid approach? LogSoftmax during training, Softmax during inference

I am training a classifier assigning one of four possible classes to each frame in a preprocessed audio stream using pytorch. I am using cross-entropy loss as the loss function for training. It is ...
0
votes
1answer
14 views

How to pad a batch of documents?

Hello PyTorch experts: Sentences and documents both can be variable length. Lets say, we have following 2 docs: ...
2
votes
0answers
17 views

How is it possible to upsample 2x with a 3x3 convolution?

From the Pytorch docs on Conv2Transpose2d, the formula to compute the output of the upsampled convolution (assuming square input and no kernel dilation) is: $$H_{out} = (H_{in} - 1) \times S - 2P_{in}+...
1
vote
0answers
17 views

Learning the distribution of a continuous variable using LSTM

I am trying to implement the following paper : https://arxiv.org/pdf/2006.10701.pdf. In order, to estimate the priors of the hidden states which have continuous values, the authors use a LSTM. I have ...
0
votes
0answers
16 views

How to use transformed parameters in pytorch model (Module)?

(This question is also posted on https://discuss.pytorch.org/t/how-to-use-transformed-parameters-in-pytorch-model-module/92984?u=adamr . With permission, I will propagate here any answers I would get ...
0
votes
0answers
28 views

How to insert sentences into training data which has 2 words, 3 words 4 and 5 etc into training data?

I have a set of sentences which each contain 2 words, 3 words, 4 words, 5 words etc. When I am trying to give the training data only the first two words in a sentence it is not accepting it. It is ...
0
votes
0answers
23 views

PyTorch Hugging Face - Language generation with torchscript model

Have a fine-tuned a summarization model following the Hugging Face seq2seq guide (starting from sshleifer/distilbart-xsum-12-6). We are interested in using AWS elastic inference for deployment for ...
1
vote
0answers
13 views

Text generation - Input text (one sentence or many sentences)

I am currently working on a project: I want to generate text with a LSTM using Pytorch. My model is working but I have a question about the methodology: I'm using the BPTTIterator and something seems ...
0
votes
0answers
20 views

Do learning rate scheduler have any significant improvement or redundant on Adam optimizer?

As in paper, Adam optimizer is adaptive learning rates algorithm. Is learning rate scheduler become redundant when use with Adam and AdamW ? Is it best practices to use learning rate scheduler with ...
0
votes
0answers
21 views

Training loss is increasing while testing loss is decreasing and Accuracy stands still

I am following the Udacity Intro to Pytorch on Exercise 7 which is to make a model which can recognize dogs or cats. Here is my code. ...
-1
votes
1answer
21 views

Bidirectional vs. Traditional LSTM [closed]

I'm working on image captioning problem, where I need to have an encoder for image and decoder for caption generation. Regarding the decoder, I've found a reference that uses Pytorch LSTM where ...
0
votes
1answer
14 views

Loading a Model with weights and optimizers without creating an instance in PyTorch

I recently downloaded Camembert Model to fine-tune it for my purpose. Upon unzipping the file the contents are: Upon loading the model.pt file using pytorch: ...
0
votes
0answers
29 views

Why the training loss increases, and predict everything as '1' or '0'

Those two pictures are from two similar experiments using same code. I am fine-tuning a pretrained-Bert model to do a binary text classification task, the dataset is 50% positive vs 50% negative, so ...
2
votes
1answer
27 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
1
vote
1answer
17 views

pytorch convolution with 0-stride along one dimension

For some square images, I'd like to use torch.nn.Conv2d with the kernel as a vertical block. As in, the kernel size is defined as max value of the first dimension by 1. Since the first dimension has ...
0
votes
0answers
7 views

Generating HD images - low cost options

I have read online about ways to create HD images using Deep learning. StyleGAN is the the often quoted one. But it very expensive to train on new set of images. It takes around 14 days to 60 days as ...
1
vote
0answers
27 views

Why does the learning rate influence whether i get a error from BCE or not?

When I use a learning rate higher than 0.001, I get this: Assertion `input_val >= zero && input_val <= one` failed. This means that the input I gave ...
0
votes
1answer
27 views

Passing Dependency/Constituency trees to a Neural Machine Translator

I am working on a project on Neural Machine Translation in the English-Irish domain. I am not an expert and have researched entirely on my own for a technology exhibition so apologies if my question ...
2
votes
1answer
41 views

Predicting sequence element based on the previous M and the following N elements

I have an array of sequences of equal length, each sequence contains 300 numbers (M=300). Each element in a sequence is a number from 1 to 9: ...
1
vote
1answer
58 views

How to reduce overfitting in a pre-trained network

I have a custom dataset with 10 classes and I am using a pre-trained resnet18 model from torch-vision. I can clearly see it's over-fitting because: the model is trained for 75 epochs with a batch size ...
0
votes
1answer
23 views

Multiclassification with large number of labels

I am attempting to build a classifier with a large input space of one hot encoded vectors. The output should be a vector of labels, with 10000 possible labels each. For example, the labels could ...
3
votes
1answer
27 views

Can I install Tensorflow and Keras on Cloud?

I will like to install Tensorflow and Keras on my PC. I use 32 bits OS. I learnt Tensorflow is not compatible with 32 bits. I cannot upgrade my OS to 64 bits since my hardware does not support it. I ...
3
votes
0answers
30 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
2
votes
2answers
40 views

How to make a neural network generalizes better?

I designed a neural network model with large number of output predicted by softmax function. However, I want categorize all the outputs into 5 outputs without modifying the architecture of other ...
1
vote
1answer
61 views

Where is the Backward function defined in PyTorch?

This might sound a little basic but while running the code below, I wanted to see the source code of the backward function: ...
0
votes
0answers
16 views

How do I efficiently load data from disk during training of deep learning models in pytorch?

I'm trying to train a deep learning model without loading the entire dataset into memory. My main question is, what's the best way of doing this? It seems like HDF5 is a common method that people ...
0
votes
0answers
16 views

Transfer learning by using vgg in pytorch

I am using vgg16 for image classification. I want to test my transfered model with the following code: ...
0
votes
0answers
34 views

What is the difference between register_buffer() and parameter.detach() in PyTorch?

I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
1
vote
1answer
34 views
0
votes
0answers
18 views

Concatenating Encoder hidden states in LSTM pytorch

I am implementing a seq2seq autoencoder in pytorch: Q1) While it is true that we can keep the encoder as bidirectional, but can we keep the decoder as bidirectional as well(does it make any sense) if ...
0
votes
1answer
40 views

Why PyTorch is faster than sklearn models?

Recently, I get to know about the hummingbird library for Python. I trained a RandomForest on a 10M-sized dataset with 2 labels. With sklearn it was taking 450 ms for inference. But after converting ...
1
vote
1answer
33 views

Classifier using pytorch

I'm writing a demo code to predict a 2-class classification for a dataset of 10-D inputs. Below, function _data generates the data: ...

1
2 3 4 5
7