Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

361 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
6 votes
0 answers
611 views

Adversarial Learning for Semantic Segmentation

I am incorporating Adversarial Training for Semantic Segmentation from Adversarial Learning for Semi-Supervised Semantic Segmentation. The idea is like this: The discriminator takes as input a ...
ethelion's user avatar
3 votes
1 answer
242 views

How is padding masking considered in the Attention Head of a Transformer?

For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
Christian's user avatar
  • 171
3 votes
0 answers
237 views

Struggling to understand/implement Transformer Decoder

I'm struggling to understand the decoder in a Transformer model, specifically with regards to some aspects of its architecture as well as how it actually handles the data during training. What I have ...
cuuupid's user avatar
  • 131
3 votes
0 answers
737 views

What exactly negative/positive value of Captum's Integrated Gradient mean?

I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ... Positive attribution score means that the input in that ...
3ORZ's user avatar
  • 31
3 votes
0 answers
898 views

PyTorch: Train without dataloader (loop trough dataframe instead)

I was wondering if it is bad practice to instead of using built in tools such as dataloader just loop trough each row in a pandas df. Lets say I am doing text classification and my training loop looks ...
Isbister's user avatar
  • 183
3 votes
1 answer
139 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
truth's user avatar
  • 280
3 votes
0 answers
124 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
Begoodpy's user avatar
  • 223
3 votes
0 answers
676 views

Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: ...
Mikhail_Sam's user avatar
3 votes
1 answer
291 views

Policy Gradient not "learning"

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
Harpal's user avatar
  • 913
3 votes
1 answer
237 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
Javier Jiménez de la Jara's user avatar
3 votes
0 answers
776 views

Understanding depthwise convolution vs convolution with group parameters in pytorch

So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows. For a input feature map of (C_in, F_in, F_in), we take only 1 ...
lincr's user avatar
  • 91
3 votes
1 answer
260 views

Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?

I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
Josh Sharkey's user avatar
3 votes
0 answers
302 views

Why embedding or rnn/lstm can not handle variable length sequence?

Pytorch embedding or lstm (I don't know about other dnn libraries) can not handle variable-length sequence by default. I am seeing various hacks to handle variable length. But my question is, why this ...
sovon's user avatar
  • 521
3 votes
0 answers
254 views

static graphs v.s. dynamic graphs

In summary, static graphs are easy to optimize but lack the expressivity found in higher-level languages; dynamic graphs provide this missing expressivity but introduce new compilation and execution ...
skytree's user avatar
  • 131
2 votes
0 answers
1k views

How to implement an RBF network in Pytorch?

This is how I understand an RBF network. From a set of points $P:=\{p_i\}_{i \in I}$ and values $F:=\{f_i\}_{i \in I}$, an RBF network is an approximation $\tilde{f}(x)$, $$\tilde{f}(x) = \sum_{j \in ...
tmaric's user avatar
  • 160
2 votes
0 answers
132 views

how Can we add extra word embedding to the pytorch funnel transformer?

i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow. first i created laserembedding like this : ...
Syed Mobassir's user avatar
2 votes
1 answer
2k views

Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same

I am trying to run this notebook on Apple M1 (1st gen) running MacOS 12.4, libs freeze: ...
Deepkamal's user avatar
  • 121
2 votes
0 answers
275 views

How to train a Task Specific Knowledge Distillation model using Hugging face model

I was referring to this code: https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb From @philschmid I could follow most of the ...
MAC's user avatar
  • 277
2 votes
0 answers
215 views

Cluster tabular data with text in some columns

Let's say I have a following features in the my dataframe: user_id user_age is_student is_graduate salary resume integer integer binary binary integer text (up to 1000 symbols) And also a few more ...
Mike's user avatar
  • 21
2 votes
0 answers
61 views

Best practices to train a transformer text classifier to predict/handle unseen labels

I fine-tuned a RoBERTa sequence classifier to classify paragraphs of certain documents using labeled paragraphs only (and skipping paragraphs with no label given). The model was validated and tested ...
Arthuro's user avatar
  • 91
2 votes
0 answers
569 views

Linear regression with Pytorch not converging

I am trying to perform a simple linear regression using Pytorch lightning (a network with only one neuron). The network is supposed to learn a simple function: y=-4x...
erap129's user avatar
  • 121
2 votes
1 answer
132 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
SpaceCossack's user avatar
2 votes
0 answers
106 views

Methods to visualize the filters in the later layers of a CNN?

I've extracted the weights from the filters of a pretrained model (AlexNet). I wish to represent these weights visually, this works fine for the first layer as there is only 3 input channels so I can ...
Fraser Hamilton's user avatar
2 votes
0 answers
87 views

Time series forecasting. How use future values

I have a time series dataset containing hourly data from a few year, like below. Let's assume that I want to make prediction for the next 3 hours (2021-01-01 19:00, 2021-01-01 20:00, 2021-01-01 21:00)....
Lazer's user avatar
  • 21
2 votes
0 answers
796 views

Train and valid accuracy and loss stay the same over many epochs of training with Pytorch

I am building a neural network to recognize hand gestures from the leapGestRecog data set from Kaggle. While training I ran into some issues. Here are some images of my data set: I augmented my data ...
Mariusz's user avatar
  • 21
2 votes
1 answer
194 views

Why yolo4 pytorch re-training loss seems high as like first time training?

I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git. I generated checkpoints by giving ...
Rajesh das's user avatar
2 votes
0 answers
338 views

How is it possible to upsample 2x with a 3x3 convolution?

From the Pytorch docs on Conv2Transpose2d, the formula to compute the output of the upsampled convolution (assuming square input and no kernel dilation) is: $$H_{out} = (H_{in} - 1) \times S - 2P_{in}+...
A is for Ambition's user avatar
2 votes
0 answers
201 views

Do learning rate scheduler have any significant improvement or redundant on Adam optimizer?

As in paper, Adam optimizer is adaptive learning rates algorithm. Is learning rate scheduler become redundant when use with Adam and AdamW ? Is it best practices to use learning rate scheduler with ...
Haha TTpro's user avatar
2 votes
0 answers
583 views

What is the difference between register_buffer() and parameter.detach() in PyTorch?

I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
Jason Young's user avatar
2 votes
1 answer
205 views

What's an appropriate datastore for variable length sequence data for PyTorch consumption?

I have a large number of sequences - potentially hundreds of thousands - each consisting of between 100 and 10,000 items, which each consist of about 5 floats. I need a datastore that can rapidly ...
Sam's user avatar
  • 121
2 votes
0 answers
329 views

PyTorch equivalent of tf.Data

I've created a pipeline using tf.Data (or more accurately a mix of Pandas and then tf.Data). ...
David Waterworth's user avatar
2 votes
0 answers
852 views

How can I get testing accuracy using tensorboard for Detectron2?

I'm learning to use Detecron2. I've followed this link to create a custom object detector. My training code - ...
mefahimrahman's user avatar
2 votes
1 answer
177 views

Row-wise Jacobian with pytorch

Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$. Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$. I would like to ...
Sam's user avatar
  • 121
2 votes
0 answers
38 views

Object coordinate detection with capsNet

Where can I find the implementation of object coordinate extraction with capsule networks in keras or pytorch?
Moeinh77's user avatar
  • 221
2 votes
0 answers
46 views

hidden state of each sequence of mini-batch

I am new to Pytorch and trying to implement a lstm character level seq2seq model. What I am trying to do is: Each sequence is a list of the characters of a particular word and several words will ...
sovon's user avatar
  • 521
2 votes
0 answers
306 views

Pytorch: How to implement nested transformers: a character-level transformer for words and a word-level transformer for sentences?

I have a model in mind, but I'm having a hard time figuring out how to actually code it in Pytorch, especially when it comes to training the model (e.g. how to define mini-batches, etc.). First of all ...
Pablo Messina's user avatar
2 votes
0 answers
1k views

LSTM not converging

I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
Bill's user avatar
  • 21
2 votes
0 answers
278 views

Moving to pytorch from tensorflow: practical considerations regarding inputs

As TF 2.0 looms and with it the certainty of having to rewrite or throw away most of my scripts, I am considering switching to pytorch. I initially liked TF for its low-level API — I think it is ...
user209974's user avatar
2 votes
0 answers
858 views

Pytorch: How to create an update rule the doesn't come from derivatives?

I want to implement the following algorithm, taken from this book, section 13.6: Here, the neural networks' outputs are $V(S, w)$ and $\pi(A|S,\theta)$, parameterized by $w$ and $\theta$ respectively....
Gulzar's user avatar
  • 196
2 votes
1 answer
110 views

Word2vec compact models

Tell me if there are any w2v models that do not require a dictionary. So, everything that I found in torchtext first wants to know the dictionary build_vocab. But if I have a huge body of text, I ...
hedgehogues's user avatar
2 votes
0 answers
925 views

get a can't set attribute while using GPU in google colab but not not while using CPU

Hi i was learning to create a classifier using pytorch in google colab that i learned in Udacity. here is the link so i was loading data in the dataloader and when i used cpu it loaded and displayed ...
Amarnath's user avatar
  • 351
2 votes
2 answers
1k views

Combining different features as input to Neural Network

I use two different sources of information as input to my neural model. The model takes a word as input and produces binary [1/0] output. I have represented each word by using its word embedding (1024 ...
zwlayer's user avatar
  • 259
2 votes
1 answer
214 views

Policy gradient - and auto-differentiation (Pytorch/Tensorflow)

In policy gradient, we have something like this: Is my understanding correct that if I apply log cross-entropy on the last layer, the gradient will be automatically calculated as per formula above?
Jed's user avatar
  • 21
2 votes
2 answers
653 views

Loss function when the output is a single probability

I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1] ...
Juan Leni's user avatar
  • 1,059
2 votes
0 answers
647 views

Output size is too small for SpatialAveragePooling in Unet

Tried this to use Resnext as encoder in Unet from here, but keep getting RuntimeError: Given input size: (4320x4x4). Calculated output size: (4320x-6x-6). Output size is too small at /opt/conda/conda-...
Rocketq's user avatar
  • 143
2 votes
1 answer
2k views

Enable Mini-batch Processing on PyTorch Word Embeddings

I am new to PyTorch and trying to create word embeddings. I started with the example below and everything works fine and it completes relatively quickly. ...
Skiddles's user avatar
  • 988
1 vote
0 answers
18 views

word2vec predicts the same word for all inputs

i build word2vec network with 2 linear layers from pytorch. for every word as an input i consistently train model to predict words before and after, for example: i was visiting my grandma's house, for ...
Тима 's user avatar
1 vote
0 answers
29 views

Gaussian process regression not working in GPytorch and Scikit-learn, can't find suitable hyperparameters

This is a MWE of my problem, basically I want to find out the map between qin and qout using a Gaussian process and with that ...
Hans's user avatar
  • 111
1 vote
0 answers
49 views

Negative MNIST - Trying to understand what's going on here

I'm new to machine learning and pytorch and any insight would be greatly appreciated. I've been playing around with the MNIST number recognition dataset and thought it would be interesting to train ...
Sloth Man's user avatar
  • 111
1 vote
0 answers
26 views

How to apply this schematic of a CNN?

I'm trying to apply a model from a paper to my problem, however I get very poor results (R² = 0.1 for regression). I think I don't understand the schematic drawing of the CNN used in the paper. Could ...
YPOC's user avatar
  • 111

1
2 3 4 5
8