Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
369
questions with no upvoted or accepted answers
6
votes
0
answers
599
views
Adversarial Learning for Semantic Segmentation
I am incorporating Adversarial Training for Semantic Segmentation from Adversarial Learning for Semi-Supervised Semantic Segmentation.
The idea is like this:
The discriminator takes as input a ...
- 61
3
votes
1
answer
88
views
How is padding masking considered in the Attention Head of a Transformer?
For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
- 171
3
votes
1
answer
232
views
Differentiable approximation for counting negative values in array
I have an array of time of arrivals and I want to convert it to count data using pytorch in a differentiable way.
Example arrival times:
...
3
votes
0
answers
173
views
Struggling to understand/implement Transformer Decoder
I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training.
What I have ...
- 131
3
votes
0
answers
516
views
What exactly negative/positive value of Captum's Integrated Gradient mean?
I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ...
Positive attribution score means that the input in that ...
- 31
3
votes
0
answers
733
views
PyTorch: Train without dataloader (loop trough dataframe instead)
I was wondering if it is bad practice to instead of using built in tools such as dataloader just loop trough each row in a pandas df. Lets say I am doing text classification and my training loop looks ...
- 173
3
votes
1
answer
130
views
How to specify version for dependencies so that each one is compatible and stays within a size limit?
I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB.
I am using my resnet34 model as a .pkl file.
I create model with it using the fastai ...
- 280
3
votes
0
answers
111
views
AlexNet Research Paper VS PytTorch and Tensorflow implementation
I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain.
In the research paper, ...
- 223
3
votes
0
answers
591
views
Explain FastText model using SHAP values
I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: ...
- 131
3
votes
1
answer
178
views
Policy Gradient not "learning"
I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
- 903
3
votes
1
answer
178
views
Is it possible to solve Rubik's cube using DQN?
I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results:
Loss goes up and ...
3
votes
0
answers
596
views
Understanding depthwise convolution vs convolution with group parameters in pytorch
So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows.
For a input feature map of (C_in, F_in, F_in), we take only 1 ...
- 91
3
votes
1
answer
232
views
Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?
I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
- 31
3
votes
0
answers
287
views
Why embedding or rnn/lstm can not handle variable length sequence?
Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
- 521
3
votes
0
answers
223
views
static graphs v.s. dynamic graphs
In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
- 131
2
votes
1
answer
71
views
Ways to share Pytorch model without revealing architecture?
We are trying to give a model to collaborators but would like to protect the IP. What are some ways to encrypt/hide/compile the definition when sharing a trained model?
- 21
2
votes
0
answers
95
views
how Can we add extra word embedding to the pytorch funnel transformer?
i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow.
first i created laserembedding like this :
...
- 121
2
votes
0
answers
1k
views
Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same
I am trying to run this notebook on Apple M1 (1st gen) running MacOS 12.4,
libs freeze:
...
- 121
2
votes
0
answers
169
views
How to train a Task Specific Knowledge Distillation model using Hugging face model
I was referring to this code:
https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb
From @philschmid
I could follow most of the ...
- 277
2
votes
0
answers
127
views
Cluster tabular data with text in some columns
Let's say I have a following features in the my dataframe:
user_id
user_age
is_student
is_graduate
salary
resume
integer
integer
binary
binary
integer
text (up to 1000 symbols)
And also a few more ...
- 21
2
votes
0
answers
45
views
Best practices to train a transformer text classifier to predict/handle unseen labels
I fine-tuned a RoBERTa sequence classifier to classify paragraphs of certain documents using labeled paragraphs only (and skipping paragraphs with no label given). The model was validated and tested ...
- 31
2
votes
0
answers
430
views
Linear regression with Pytorch not converging
I am trying to perform a simple linear regression using Pytorch lightning (a network with only one neuron). The network is supposed to learn a simple function: y=-4x...
- 121
2
votes
0
answers
55
views
When to stop the final model training?
Let's say I'm participating in a Kaggle image recognition competition.
Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
- 21
2
votes
0
answers
95
views
Methods to visualize the filters in the later layers of a CNN?
I've extracted the weights from the filters of a pretrained model (AlexNet). I wish to represent these weights visually, this works fine for the first layer as there is only 3 input channels so I can ...
2
votes
0
answers
75
views
Time series forecasting. How use future values
I have a time series dataset containing hourly data from a few year, like below. Let's assume that I want to make prediction for the next 3 hours (2021-01-01 19:00, 2021-01-01 20:00, 2021-01-01 21:00)....
- 21
2
votes
0
answers
727
views
Train and valid accuracy and loss stay the same over many epochs of training with Pytorch
I am building a neural network to recognize hand gestures from the leapGestRecog data set from Kaggle. While training I ran into some issues.
Here are some images of my data set:
I augmented my data ...
- 21
2
votes
1
answer
170
views
Why yolo4 pytorch re-training loss seems high as like first time training?
I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git.
I generated checkpoints by giving ...
- 113
2
votes
0
answers
295
views
How is it possible to upsample 2x with a 3x3 convolution?
From the Pytorch docs on Conv2Transpose2d, the formula to compute the output of the upsampled convolution (assuming square input and no kernel dilation) is:
$$H_{out} = (H_{in} - 1) \times S - 2P_{in}+...
2
votes
0
answers
177
views
Do learning rate scheduler have any significant improvement or redundant on Adam optimizer?
As in paper, Adam optimizer is adaptive learning rates algorithm.
Is learning rate scheduler become redundant when use with Adam and AdamW ?
Is it best practices to use learning rate scheduler with ...
- 233
2
votes
0
answers
511
views
What is the difference between register_buffer() and parameter.detach() in PyTorch?
I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
- 21
2
votes
1
answer
182
views
What's an appropriate datastore for variable length sequence data for PyTorch consumption?
I have a large number of sequences - potentially hundreds of thousands - each consisting of between 100 and 10,000 items, which each consist of about 5 floats.
I need a datastore that can rapidly ...
- 121
2
votes
0
answers
312
views
PyTorch equivalent of tf.Data
I've created a pipeline using tf.Data (or more accurately a mix of Pandas and then tf.Data). ...
- 400
2
votes
0
answers
724
views
How can I get testing accuracy using tensorboard for Detectron2?
I'm learning to use Detecron2. I've followed this link to create a custom object detector.
My training code -
...
2
votes
1
answer
164
views
Row-wise Jacobian with pytorch
Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$.
Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$.
I would like to ...
- 121
2
votes
0
answers
36
views
Object coordinate detection with capsNet
Where can I find the implementation of object coordinate extraction with capsule networks in keras or pytorch?
- 223
2
votes
0
answers
42
views
hidden state of each sequence of mini-batch
I am new to Pytorch and trying to implement a lstm character level seq2seq model.
What I am trying to do is:
Each sequence is a list of the characters of a particular word and several words will ...
- 521
2
votes
1
answer
413
views
A model that only works by setting all initial weights to zero
In this model from MusicNet, they set the initial weights of their neural network to all zeros.
...
- 123
2
votes
0
answers
285
views
Pytorch: How to implement nested transformers: a character-level transformer for words and a word-level transformer for sentences?
I have a model in mind, but I'm having a hard time figuring out how to actually code it in Pytorch, especially when it comes to training the model (e.g. how to define mini-batches, etc.). First of all ...
- 377
2
votes
0
answers
1k
views
LSTM not converging
I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
- 21
2
votes
0
answers
267
views
Moving to pytorch from tensorflow: practical considerations regarding inputs
As TF 2.0 looms and with it the certainty of having to rewrite or throw away most of my scripts, I am considering switching to pytorch. I initially liked TF for its low-level API — I think it is ...
- 131
2
votes
0
answers
806
views
Pytorch: How to create an update rule the doesn't come from derivatives?
I want to implement the following algorithm, taken from this book, section 13.6:
Here, the neural networks' outputs are $V(S, w)$ and $\pi(A|S,\theta)$, parameterized by $w$ and $\theta$ respectively....
- 196
2
votes
0
answers
878
views
get a can't set attribute while using GPU in google colab but not not while using CPU
Hi i was learning to create a classifier using pytorch in google colab that i learned in Udacity. here is the link
so i was loading data in the dataloader and when i used cpu it loaded and displayed ...
- 351
2
votes
2
answers
1k
views
Combining different features as input to Neural Network
I use two different sources of information as input to my neural model. The model takes a word as input and produces binary [1/0] output. I have represented each word by using its word embedding (1024 ...
- 229
2
votes
1
answer
192
views
Policy gradient - and auto-differentiation (Pytorch/Tensorflow)
In policy gradient, we have something like this:
Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
- 21
2
votes
2
answers
575
views
Loss function when the output is a single probability
I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1]
...
- 1,039
2
votes
0
answers
639
views
Output size is too small for SpatialAveragePooling in Unet
Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
- 143
2
votes
1
answer
2k
views
Enable Mini-batch Processing on PyTorch Word Embeddings
I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly.
...
- 978
1
vote
1
answer
28
views
Understanding batch size, sequence, sequence length and batch length of a RNN
My Problem
I'm struggling with the different definitions of batch size, sequence, sequence length and batch length of a RNN and how to use it in the correct way.
First things first - let's clarify the ...
- 11
1
vote
1
answer
38
views
Convolution neural network loss increasing instead of decreasing
I am working on a binary image classification task in which I have greyscale images of size (1, 224, 224) (all normalized between 0 and 1) and a set of labels (0 or 1). I have around 2.6k images with ...
1
vote
1
answer
109
views
Is it possible to modify YOLOv8 to use it as a feature extractor for other tasks?
I'm reading through the documentation of YOLOv8 here, but I fail to see an easy way to do what I suggest in the title. What I want to do is to load a pretrained YOLOv8 model, create a bigger model ...
- 377