Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

356 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
0 answers
606 views

Adversarial Learning for Semantic Segmentation

I am incorporating Adversarial Training for Semantic Segmentation from Adversarial Learning for Semi-Supervised Semantic Segmentation. The idea is like this: The discriminator takes as input a ...
ethelion's user avatar
3 votes
1 answer
210 views

How is padding masking considered in the Attention Head of a Transformer?

For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
Christian's user avatar
  • 171
3 votes
0 answers
229 views

Struggling to understand/implement Transformer Decoder

I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training. What I have ...
cuuupid's user avatar
  • 131
3 votes
0 answers
706 views

What exactly negative/positive value of Captum's Integrated Gradient mean?

I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ... Positive attribution score means that the input in that ...
3ORZ's user avatar
  • 31
3 votes
0 answers
877 views

PyTorch: Train without dataloader (loop trough dataframe instead)

I was wondering if it is bad practice to instead of using built in tools such as dataloader just loop trough each row in a pandas df. Lets say I am doing text classification and my training loop looks ...
Isbister's user avatar
  • 183
3 votes
1 answer
137 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
truth's user avatar
  • 280
3 votes
0 answers
120 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
Begoodpy's user avatar
  • 223
3 votes
0 answers
667 views

Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: ...
Mikhail_Sam's user avatar
3 votes
1 answer
267 views

Policy Gradient not "learning"

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
Harpal's user avatar
  • 913
3 votes
1 answer
224 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
Javier Jiménez de la Jara's user avatar
3 votes
0 answers
750 views

Understanding depthwise convolution vs convolution with group parameters in pytorch

So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows. For a input feature map of (C_in, F_in, F_in), we take only 1 ...
lincr's user avatar
  • 91
3 votes
1 answer
252 views

Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?

I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
Josh Sharkey's user avatar
3 votes
0 answers
298 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
sovon's user avatar
  • 521
3 votes
0 answers
248 views

static graphs v.s. dynamic graphs

In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
skytree's user avatar
  • 131
2 votes
0 answers
1k views

How to implement an RBF network in Pytorch?

This is how I understand an RBF network. From a set of points $P:=\{p_i\}_{i \in I}$ and values $F:=\{f_i\}_{i \in I}$, an RBF network is an approximation $\tilde{f}(x)$, $$\tilde{f}(x) = \sum_{j \in ...
tmaric's user avatar
  • 160
2 votes
0 answers
128 views

how Can we add extra word embedding to the pytorch funnel transformer?

i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow. first i created laserembedding like this : ...
Syed Mobassir's user avatar
2 votes
0 answers
2k views

Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same

I am trying to run this notebook on Apple M1 (1st gen) running MacOS 12.4, libs freeze: ...
Deepkamal's user avatar
  • 121
2 votes
0 answers
268 views

How to train a Task Specific Knowledge Distillation model using Hugging face model

I was referring to this code: https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb From @philschmid I could follow most of the ...
MAC's user avatar
  • 277
2 votes
0 answers
209 views

Cluster tabular data with text in some columns

Let's say I have a following features in the my dataframe: user_id user_age is_student is_graduate salary resume integer integer binary binary integer text (up to 1000 symbols) And also a few more ...
Mike's user avatar
  • 21
2 votes
0 answers
59 views

Best practices to train a transformer text classifier to predict/handle unseen labels

I fine-tuned a RoBERTa sequence classifier to classify paragraphs of certain documents using labeled paragraphs only (and skipping paragraphs with no label given). The model was validated and tested ...
Arthuro's user avatar
  • 91
2 votes
0 answers
548 views

Linear regression with Pytorch not converging

I am trying to perform a simple linear regression using Pytorch lightning (a network with only one neuron). The network is supposed to learn a simple function: y=-4x...
erap129's user avatar
  • 121
2 votes
1 answer
111 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
SpaceCossack's user avatar
2 votes
0 answers
104 views

Methods to visualize the filters in the later layers of a CNN?

I've extracted the weights from the filters of a pretrained model (AlexNet). I wish to represent these weights visually, this works fine for the first layer as there is only 3 input channels so I can ...
Fraser Hamilton's user avatar
2 votes
0 answers
85 views

Time series forecasting. How use future values

I have a time series dataset containing hourly data from a few year, like below. Let's assume that I want to make prediction for the next 3 hours (2021-01-01 19:00, 2021-01-01 20:00, 2021-01-01 21:00)....
Lazer's user avatar
  • 21
2 votes
0 answers
791 views

Train and valid accuracy and loss stay the same over many epochs of training with Pytorch

I am building a neural network to recognize hand gestures from the leapGestRecog data set from Kaggle. While training I ran into some issues. Here are some images of my data set: I augmented my data ...
Mariusz's user avatar
  • 21
2 votes
1 answer
189 views

Why yolo4 pytorch re-training loss seems high as like first time training?

I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git. I generated checkpoints by giving ...
Rajesh das's user avatar
2 votes
0 answers
333 views

How is it possible to upsample 2x with a 3x3 convolution?

From the Pytorch docs on Conv2Transpose2d, the formula to compute the output of the upsampled convolution (assuming square input and no kernel dilation) is: $$H_{out} = (H_{in} - 1) \times S - 2P_{in}+...
A is for Ambition's user avatar
2 votes
0 answers
196 views

Do learning rate scheduler have any significant improvement or redundant on Adam optimizer?

As in paper, Adam optimizer is adaptive learning rates algorithm. Is learning rate scheduler become redundant when use with Adam and AdamW ? Is it best practices to use learning rate scheduler with ...
Haha TTpro's user avatar
2 votes
0 answers
571 views

What is the difference between register_buffer() and parameter.detach() in PyTorch?

I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
Jason Young's user avatar
2 votes
1 answer
200 views

What's an appropriate datastore for variable length sequence data for PyTorch consumption?

I have a large number of sequences - potentially hundreds of thousands - each consisting of between 100 and 10,000 items, which each consist of about 5 floats. I need a datastore that can rapidly ...
Sam's user avatar
  • 121
2 votes
0 answers
327 views

PyTorch equivalent of tf.Data

I've created a pipeline using tf.Data (or more accurately a mix of Pandas and then tf.Data). ...
David Waterworth's user avatar
2 votes
0 answers
839 views

How can I get testing accuracy using tensorboard for Detectron2?

I'm learning to use Detecron2. I've followed this link to create a custom object detector. My training code - ...
mefahimrahman's user avatar
2 votes
1 answer
173 views

Row-wise Jacobian with pytorch

Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$. Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$. I would like to ...
Sam's user avatar
  • 121
2 votes
0 answers
36 views

Object coordinate detection with capsNet

Where can I find the implementation of object coordinate extraction with capsule networks in keras or pytorch?
Moeinh77's user avatar
  • 221
2 votes
0 answers
44 views

hidden state of each sequence of mini-batch

I am new to Pytorch and trying to implement a lstm character level seq2seq model. What I am trying to do is: Each sequence is a list of the characters of a particular word and several words will ...
sovon's user avatar
  • 521
2 votes
0 answers
303 views

Pytorch: How to implement nested transformers: a character-level transformer for words and a word-level transformer for sentences?

I have a model in mind, but I'm having a hard time figuring out how to actually code it in Pytorch, especially when it comes to training the model (e.g. how to define mini-batches, etc.). First of all ...
Pablo Messina's user avatar
2 votes
0 answers
1k views

LSTM not converging

I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
Bill's user avatar
  • 21
2 votes
0 answers
276 views

Moving to pytorch from tensorflow: practical considerations regarding inputs

As TF 2.0 looms and with it the certainty of having to rewrite or throw away most of my scripts, I am considering switching to pytorch. I initially liked TF for its low-level API — I think it is ...
user209974's user avatar
2 votes
0 answers
850 views

Pytorch: How to create an update rule the doesn't come from derivatives?

I want to implement the following algorithm, taken from this book, section 13.6: Here, the neural networks' outputs are $V(S, w)$ and $\pi(A|S,\theta)$, parameterized by $w$ and $\theta$ respectively....
Gulzar's user avatar
  • 196
2 votes
0 answers
923 views

get a can't set attribute while using GPU in google colab but not not while using CPU

Hi i was learning to create a classifier using pytorch in google colab that i learned in Udacity. here is the link so i was loading data in the dataloader and when i used cpu it loaded and displayed ...
Amarnath's user avatar
  • 351
2 votes
2 answers
1k views

Combining different features as input to Neural Network

I use two different sources of information as input to my neural model. The model takes a word as input and produces binary [1/0] output. I have represented each word by using its word embedding (1024 ...
zwlayer's user avatar
  • 259
2 votes
1 answer
209 views

Policy gradient - and auto-differentiation (Pytorch/Tensorflow)

In policy gradient, we have something like this: Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
Jed's user avatar
  • 21
2 votes
2 answers
637 views

Loss function when the output is a single probability

I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1] ...
Juan Leni's user avatar
  • 1,059
2 votes
0 answers
643 views

Output size is too small for SpatialAveragePooling in Unet

Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
Rocketq's user avatar
  • 143
2 votes
1 answer
2k views

Enable Mini-batch Processing on PyTorch Word Embeddings

I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly. ...
Skiddles's user avatar
  • 988
1 vote
1 answer
93 views

Neural Net not able to learn simple analytical equation

I am currently making my first attempts with Pytorch. I am trying to solve a simple equation with a neural net. Analytically solved, the result of my neural net shall look like this: $$ y = \frac{x_5}{...
Ripleys's user avatar
  • 11
1 vote
0 answers
60 views

Pytorch Transformer only generating NaN when using mask

When I generate a src_mask like this ...
kot's user avatar
  • 11
1 vote
0 answers
547 views

How to implement 2d Rotary Position Embedding in PyTorch?

The original RoPE paper suggests that the Rotary Position Embedding it describes can easily be extended to two or more dimensions: 3.2.2 in https://arxiv.org/abs/2104.09864 . I'm trying to find a ...
LogicChains's user avatar
1 vote
1 answer
15 views

gnet loss is increasing, while my dnet loss is decreasing, what is wrong with the GAN?

I am developing an MIT open source GAN to generate synthetic structured health data, in the standard FHIR format. I have a running GAN, and it is using 200,000 Synthea Patient FHIR resources to train....
FHIRFLY's user avatar
  • 11
1 vote
0 answers
111 views

Backward propagation slower as epochs increase in pytorch

I'm trying to replicate TasNet paper using Pytorch Lightning, but the training time increases as the epochs increase (the first epoch takes 20 seconds, the fifth 2:30 minutes). I've used PyTorch ...
programming-duck's user avatar

1
2 3 4 5
8