Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
334 questions with no upvoted or accepted answers
6
votes
0
answers
614
views
Adversarial Learning for Semantic Segmentation
I am incorporating Adversarial Training for Semantic Segmentation from Adversarial Learning for Semi-Supervised Semantic Segmentation.
The idea is like this:
The discriminator takes as input a ...
- 61
3
votes
1
answer
33
views
RNN performing worse than random guessing on large dataset
I have to start off by saying I am 100% a beginner here.
I trained a RNN model on a 30 class dataset with over 90000 samples and it achieved less than 2% accuracy. Training the same model on a small ...
- 31
3
votes
1
answer
332
views
How is padding masking considered in the Attention Head of a Transformer?
For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
- 181
3
votes
0
answers
244
views
Cluster tabular data with text in some columns
Let's say I have a following features in the my dataframe:
user_id
user_age
is_student
is_graduate
salary
resume
integer
integer
binary
binary
integer
text (up to 1000 symbols)
And also a few more ...
- 31
3
votes
0
answers
268
views
Struggling to understand/implement Transformer Decoder
I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training.
What I have ...
- 131
3
votes
0
answers
829
views
What exactly negative/positive value of Captum's Integrated Gradient mean?
I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ...
Positive attribution score means that the input in that ...
- 31
3
votes
0
answers
980
views
PyTorch: Train without dataloader (loop trough dataframe instead)
I was wondering if it is bad practice to instead of using built in tools such as dataloader just loop trough each row in a pandas df. Lets say I am doing text classification and my training loop looks ...
- 193
3
votes
1
answer
141
views
How to specify version for dependencies so that each one is compatible and stays within a size limit?
I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB.
I am using my resnet34 model as a .pkl file.
I create model with it using the fastai ...
- 280
3
votes
0
answers
129
views
AlexNet Research Paper VS PytTorch and Tensorflow implementation
I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain.
In the research paper, ...
- 233
3
votes
0
answers
721
views
Explain FastText model using SHAP values
I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: ...
- 131
3
votes
1
answer
333
views
Policy Gradient not "learning"
I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
- 913
3
votes
1
answer
258
views
Is it possible to solve Rubik's cube using DQN?
I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results:
Loss goes up and ...
3
votes
0
answers
819
views
Understanding depthwise convolution vs convolution with group parameters in pytorch
So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows.
For a input feature map of (C_in, F_in, F_in), we take only 1 ...
- 91
3
votes
0
answers
882
views
How can I get testing accuracy using tensorboard for Detectron2?
I'm learning to use Detecron2. I've followed this link to create a custom object detector.
My training code -
...
- 131
3
votes
1
answer
266
views
Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?
I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
- 31
3
votes
0
answers
310
views
Why embedding or rnn/lstm can not handle variable length sequence?
Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
- 521
3
votes
0
answers
263
views
static graphs v.s. dynamic graphs
In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
- 131
2
votes
0
answers
9
views
What may not work in PyTorch when using qantized models?
I suppose not all the PyTorch tools and modules work with quantized (e.g. to int8) models. But what may not work and why?
Device and Operator Support
Quantization support is restricted to a subset ...
- 791
2
votes
0
answers
26
views
Stuck with constant loss and network not learning
I am trying to predict certain function coefficients (output: a, b) based on its curve (input: frequency_response) with the help of https://github.com/Blealtan/efficient-kan (Kolmogorov-Arnold Network)...
- 151
2
votes
1
answer
28
views
Is there a method to impute missing temperature values in time series that considers external factors?
I have 10 day temperature forecast data that is hourly initially and then every 3 hours. I would like to predict the hourly values for the full 10 days. Linear interpolation fails as sunrise and other ...
- 121
2
votes
0
answers
2k
views
How to implement an RBF network in Pytorch?
This is how I understand an RBF network.
From a set of points $P:=\{p_i\}_{i \in I}$ and values $F:=\{f_i\}_{i \in I}$, an RBF network is an approximation $\tilde{f}(x)$,
$$\tilde{f}(x) = \sum_{j \in ...
- 160
2
votes
1
answer
665
views
Can __getitem__() in a PyTorch Dataset return a random sample?
Is __getitem()__ in a PyTorch Dataset restricted to always returning the same sample for the same index? I am thinking that the ...
- 145
2
votes
1
answer
458
views
Bounding box regression without a classification task
I am using PyTorch to create a model that detects certain objects in an image. I framed my problem as a regression on bounding boxes, without any classification task whatsoever. The reasoning behind ...
- 21
2
votes
0
answers
140
views
how Can we add extra word embedding to the pytorch funnel transformer?
i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow.
first i created laserembedding like this :
...
- 121
2
votes
1
answer
2k
views
Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same
I am trying to run this notebook on Apple M1 (1st gen) running MacOS 12.4,
libs freeze:
...
- 121
2
votes
0
answers
301
views
How to train a Task Specific Knowledge Distillation model using Hugging face model
I was referring to this code:
https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb
From @philschmid
I could follow most of the ...
- 277
2
votes
0
answers
86
views
Best practices to train a transformer text classifier to predict/handle unseen labels
I fine-tuned a RoBERTa sequence classifier to classify paragraphs of certain documents using labeled paragraphs only (and skipping paragraphs with no label given). The model was validated and tested ...
- 101
2
votes
0
answers
625
views
Linear regression with Pytorch not converging
I am trying to perform a simple linear regression using Pytorch lightning (a network with only one neuron). The network is supposed to learn a simple function: y=-4x...
- 121
2
votes
1
answer
183
views
When to stop the final model training?
Let's say I'm participating in a Kaggle image recognition competition.
Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
- 21
2
votes
0
answers
109
views
Methods to visualize the filters in the later layers of a CNN?
I've extracted the weights from the filters of a pretrained model (AlexNet). I wish to represent these weights visually, this works fine for the first layer as there is only 3 input channels so I can ...
2
votes
0
answers
87
views
Time series forecasting. How use future values
I have a time series dataset containing hourly data from a few year, like below. Let's assume that I want to make prediction for the next 3 hours (2021-01-01 19:00, 2021-01-01 20:00, 2021-01-01 21:00)....
- 21
2
votes
0
answers
822
views
Train and valid accuracy and loss stay the same over many epochs of training with Pytorch
I am building a neural network to recognize hand gestures from the leapGestRecog data set from Kaggle. While training I ran into some issues.
Here are some images of my data set:
I augmented my data ...
- 21
2
votes
1
answer
220
views
Why yolo4 pytorch re-training loss seems high as like first time training?
I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git.
I generated checkpoints by giving ...
- 113
2
votes
0
answers
362
views
How is it possible to upsample 2x with a 3x3 convolution?
From the Pytorch docs on Conv2Transpose2d, the formula to compute the output of the upsampled convolution (assuming square input and no kernel dilation) is:
$$H_{out} = (H_{in} - 1) \times S - 2P_{in}+...
2
votes
0
answers
231
views
Do learning rate scheduler have any significant improvement or redundant on Adam optimizer?
As in paper, Adam optimizer is adaptive learning rates algorithm.
Is learning rate scheduler become redundant when use with Adam and AdamW ?
Is it best practices to use learning rate scheduler with ...
- 243
2
votes
0
answers
617
views
What is the difference between register_buffer() and parameter.detach() in PyTorch?
I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
- 21
2
votes
0
answers
342
views
PyTorch equivalent of tf.Data
I've created a pipeline using tf.Data (or more accurately a mix of Pandas and then tf.Data). ...
- 440
2
votes
1
answer
184
views
Row-wise Jacobian with pytorch
Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$.
Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$.
I would like to ...
- 121
2
votes
0
answers
39
views
Object coordinate detection with capsNet
Where can I find the implementation of object coordinate extraction with capsule networks in keras or pytorch?
- 221
2
votes
0
answers
47
views
hidden state of each sequence of mini-batch
I am new to Pytorch and trying to implement a lstm character level seq2seq model.
What I am trying to do is:
Each sequence is a list of the characters of a particular word and several words will ...
- 521
2
votes
0
answers
317
views
Pytorch: How to implement nested transformers: a character-level transformer for words and a word-level transformer for sentences?
I have a model in mind, but I'm having a hard time figuring out how to actually code it in Pytorch, especially when it comes to training the model (e.g. how to define mini-batches, etc.). First of all ...
- 387
2
votes
0
answers
1k
views
LSTM not converging
I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
- 21
2
votes
0
answers
280
views
Moving to pytorch from tensorflow: practical considerations regarding inputs
As TF 2.0 looms and with it the certainty of having to rewrite or throw away most of my scripts, I am considering switching to pytorch. I initially liked TF for its low-level API — I think it is ...
- 131
2
votes
0
answers
869
views
Pytorch: How to create an update rule the doesn't come from derivatives?
I want to implement the following algorithm, taken from this book, section 13.6:
Here, the neural networks' outputs are $V(S, w)$ and $\pi(A|S,\theta)$, parameterized by $w$ and $\theta$ respectively....
- 196
2
votes
1
answer
118
views
Word2vec compact models
Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
- 121
2
votes
0
answers
935
views
get a can't set attribute while using GPU in google colab but not not while using CPU
Hi i was learning to create a classifier using pytorch in google colab that i learned in Udacity. here is the link
so i was loading data in the dataloader and when i used cpu it loaded and displayed ...
- 351
2
votes
1
answer
216
views
Policy gradient - and auto-differentiation (Pytorch/Tensorflow)
In policy gradient, we have something like this:
Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
- 21
2
votes
0
answers
651
views
Output size is too small for SpatialAveragePooling in Unet
Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
- 143
2
votes
1
answer
2k
views
Enable Mini-batch Processing on PyTorch Word Embeddings
I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly.
...
- 998
1
vote
0
answers
12
views
How to solve the issue with getting free ports in Pytorch DDP?
I am facing issues with getting a free port in the DDP setup block of PyTorch for parallelizing my deep learning training job across multiple GPUs on a Linux HPC cluster.
I am trying to submit a deep ...
