Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
683
questions
0
votes
0
answers
14
views
Accelerated learning when wrapping layers in a class
I am implementing a VGG-like network using Pytorch 1.13.1 (python=3.7.12) for image classification on the CINIC-10 dataset. The following two implementations turn out to have very different training ...
- 1
0
votes
1
answer
56
views
GPT-2 architecture question
I am currently working on a NLP model that compares two comments and determines which one would be more popular. I have already came up with an architecture - it will be based on GPT-2. But now I am ...
1
vote
2
answers
52
views
Neural network not learning at all
I am training a MLP on a tabular dataset, the pendigits dataset. Problem is that training loss and accuracy are more or less stable, while validation and test loss and accuracy are completely constant....
- 219
1
vote
1
answer
55
views
Class Imbalance in Dataset of Images
When dealing with an imbalanced dataset, I have been taught to oversample on only the train samples and not the entire dataset to avoid overfitting, however this was for structured text based data in ...
- 11
1
vote
1
answer
33
views
0
votes
0
answers
63
views
Reinforcement Learning (gymnasium's FrozenLake-v1) using Spiking Neural Networks (BindsNet)
I'm new to reinforcement learning. I'm trying to solve the FrozenLake-v1 game using OpenAI's gymnasium learning environment and BindsNet, which is a library to simulate Spiking Neural Networks using ...
- 1
0
votes
0
answers
20
views
In "Show, attend and tell", why do the attention weights get multiplied with the features to form the context vector?
The attention weights are formed through the last hidden state of the LSTM and the feature map from some kind of image encoder (in my case resnet so the features are in the form of 14x14x2048). They ...
- 101
0
votes
1
answer
2k
views
Pytorch mat1 and mat2 shapes cannot be multiplied
The error message shows
RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x32768 and 512x256)
I have built the following model:
...
- 27
0
votes
0
answers
22
views
Reporting and/or evaluation of metrics in deep learning
While I was trying to write a custom training script, I encountered the following doubt.
I see that the loss is evaluated at the end of every forward pass (i.e., a step or with a particular batch of ...
0
votes
0
answers
134
views
RuntimeError: Caught RuntimeError in replica x on device x. Original Traceback: ... RuntimeError: shape '[x, x, x]' is invalid for input of size xxx
I am encountering a RuntimeError with the following message:
...
0
votes
0
answers
22
views
Need help with improving validation loss and model overfitting/underfitting
I am using Ensemble PyTorch to train a voting classifier. My dataset includes around 60k records. I trained a Neural Network with Cross-entropy loss. Below is my model architecture
...
- 1
0
votes
0
answers
20
views
Avoid decrease in accuracy with depthwise convolution
I have been optimizing the model for resource-constrained IoT device, and for this, one of the automated choice was to depthwise separable convolution. However, withit, though the number of flobs, ...
0
votes
0
answers
6
views
Why does y.grad_fn(..) not return the contribution of k for y = a * x ** k?
I have this following (toy) program:
...
- 101
-1
votes
1
answer
35
views
Classifying the sum of two inputs as even or odd - Model always stuck at 50% accuracy for training and test data
I am creating a simple feed forward to classify if the sum of two inputs as even or odd.
I cannot change the input structure (has to be two nodes), and output structure (two nodes as well, one for ...
0
votes
1
answer
66
views
Visualizing dataset class distribution in pytorch
I have used in the past a seaborn plotting function to visualize the number of samples belonging to each class directory. That was very easy since each directory in which the dataset was divided ...
0
votes
0
answers
34
views
Given 20+ input numbers, predict 4-5 output numbers, using past data with the same 20+ inputs / 4-5 output for training
I'm a veteran Python software engineer, but very amateur at this stuff. I've used PyTorch for some NLP (sentiment and classification), and I've spent a bit of time attempting to learn data science, ...
- 101
0
votes
0
answers
223
views
Getting Error: TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not tuple
I am working on a CNN multi-class classification of different concentrations (10uM, 30uM, etc.) I create my dataset to include the images as the features and the concentrations as labels. Note that ...
- 3
0
votes
0
answers
69
views
is the distilRoberta transformer model overfitting or underfitting?
I am a bit new to ML, below are the results after I fine tuned distilRoBERTa using HuggingFace Trainer. I cant tell if my model is over-fitting, under-fitting or ok? I ran 7 epochs.
I think its ...
- 1
0
votes
0
answers
60
views
Multi-label classification inference
I am working on a multi-label classification with transformers. I want to assign tags to input text. First, I have trained a model multiclass and with the pipeline function I can retrieve all possible ...
- 1
0
votes
0
answers
262
views
How to implement an RBF network in Pytorch?
This is how I understand an RBF network.
From a set of points $P:=\{p_i\}_{i \in I}$ and values $F:=\{f_i\}_{i \in I}$, an RBF network is an approximation $\tilde{f}(x)$,
$$\tilde{f}(x) = \sum_{j \in ...
- 140
0
votes
0
answers
19
views
Estimate train complexity of a Pytorch model
I guess there are better "keywords" than that, especially I'm not sure about the "complexity" word. But I thought of none.
Let's say I have a NLP model with 1. a Embedding Layer 2. ...
- 311
0
votes
1
answer
129
views
Can __getitem__() in a PyTorch Dataset return a random sample?
Is __getitem()__ in a PyTorch Dataset restricted to always returning the same sample for the same index? I am thinking that the ...
- 125
0
votes
0
answers
32
views
Why are all PyTorch dataloader proccesses in S state (interruptible sleep) except one?
Short version: I'm using PyTorch dataloading library to load data in parallel for training a deep learning model. When I look at the CPU usage with htop, I see a ...
- 377
2
votes
1
answer
68
views
Ways to share Pytorch model without revealing architecture?
We are trying to give a model to collaborators but would like to protect the IP. What are some ways to encrypt/hide/compile the definition when sharing a trained model?
- 21
0
votes
0
answers
7
views
Benchmark data set for Molecular genration
I'd like to test molecular generation using Pytorch geometric, I don't have any dataset to test it. PyG group is actully providing Graph AutoEncoder and Variational GAE, ARVGAE for such purpose. I can ...
0
votes
0
answers
33
views
LightningDataModule with Trainer in PytorchLightning automatically fits validation model?
I try to fight with overfitting, this is why I decided to look through documentation (https://pytorch-lightning.readthedocs.io/en/stable/common/evaluation_basic.html#train-with-the-validation-loop), ...
0
votes
0
answers
66
views
Should i include model.train() with Dropout in PytorchLightning
I was reading guide in which an author used model.train() in each epoch because of the DropOut layer (he didn't use Pytorch Lightning). The question is - should i include model.train() in my Pytorch ...
0
votes
0
answers
21
views
How to improve pytorch network to be more precise
I have a pytorch neural network which is being trained on data containing multiple financial values and some other non numeric values translated to numbers with self created logic (for example if ...
- 101
0
votes
0
answers
382
views
How to deal with DataCollator and DataLoaders in Huggingface?
I have issues combining a DataLoader and DataCollator. The following code with DataCollatorWithPadding results in a ...
- 101
0
votes
0
answers
22
views
Object detection (with bounding boxes) end-to-end as an auxiliary task (as in multitask learning)?
Let's say that I have a visual encoder (CNN or ViT) which outputs a volume of local features of dimensions WxHxD plus a global feature vector of dimension D, which I'm currently using as the backbone ...
- 377
1
vote
2
answers
133
views
Will ML frameworks work with integrated graphics card in laptop?
I'm CS student. I'm just getting started in data science and machine learning. Can i use laptops with integrated graphics card in the data science and machine learning projects?
Will pytorch and ...
- 11
0
votes
0
answers
43
views
Transposed Convolution in PyTorch Visualisation
I am trying to visualize the output of the transpose convolution in pytorch to better understand the operation. Here is my code, inputs and output images as well:
...
- 27
0
votes
0
answers
17
views
Pruning using BERTology
I am trying out some BERT based models for a question and answering task. I need models trained on squad v2.0. To cut down on the inference time , I'm trying out pruning. I came across the BERTology ...
- 103
0
votes
0
answers
32
views
How to annotate a custom dataset to run Yolov7?
I have a personal dataset of mine on which I want to use the yolov7 network (pytorch). How do I manually annotate the dataset (assuming I know which part of the images I know to annotate) to train the ...
- 482
0
votes
1
answer
149
views
Requirements for variable length output in transformer
I have been working on modifying the transformer from the article The Annotated Transformer. One of the features I would like to include is the ability to pass a sequence of fixed length, and receive ...
- 1
0
votes
0
answers
12
views
Help with transitioning an existing DQN into a DRQN
Hi Data Science Stack Exchange community,
To preface this post, please let me know if I need to clarify any details to receive help and/or guidance. I am new to posting on Data Science Stack Exchange ...
- 1
0
votes
0
answers
9
views
AttributeError in pytorch
I want use https://github.com/mmbejani/TikhonovRegularizationTerm. This library includes the implementation of regularization tikhonov terms that were published from 2010 until now (2020).
I am trying ...
0
votes
0
answers
40
views
How do I implement recurrent activations for LSTM/GRU cells in Pytorch?
Although Tensorflow has simple parameters with which I can initialize the recurrent activation of a GRU or LSTM cell, Pytorch does not.
What is the best way to add recurrent activation in pytorch?
<...
- 164
0
votes
0
answers
79
views
Is this function how you would take the Mahalanobis Distance between tensors?
I made an attempt to copy an implementation of the Mahalanobis Distance from the PyTorch library. I'm not sure it is right or if it is more complicated than it needs to be.
I would like a working and ...
- 177
0
votes
1
answer
27
views
What is the best way to use three different losses on two classifiers?
Two classifiers need to be trained simultaneously, and I have three losses, as shown in the figure. Classifiers 1 and 2 will be updated by losses 1 and 2. Furthermore, loss 3 should update the two ...
0
votes
0
answers
85
views
How are fp32 weights converted to fp16 post training in PyTorch?
Can the weights of a model trained in full precision be converted to half precision post-training, with or without loss of accuracy? If so, what is the maths involved behind the conversion?
Thanks
- 1
0
votes
0
answers
37
views
How to replicate MMCV conv layer with integrated batch norm using PyTorch
Question
How do I replicate the following convolution layer from MMCV using PyTorch? I cannot find any reference in the MMCV docs on how norm_cfg works.
...
- 46
0
votes
1
answer
26
views
How to differentiate Bitmojis gender?
I have seen this Project Larry-zx's Githubproject
It is used to create a .pth file which can differenciate between a bitmojis gender.
Now I have created that .pth file but no idea how to use it, ...
0
votes
0
answers
115
views
How can I improve the accuracy of my pytorch neural network for classification of tabular data?
As a newbie in 'pytorch', I am building a neural network for classification of faulty water pumps in Tazania for this competition I am also using ax-platform for ...
user144514
1
vote
0
answers
18
views
Drawing transparent bounding boxes with Torchvision
I am using Torchvision in a Python script to draw bounding boxes and then crop images based on the bounding boxes drawn. The script works well, except that the bounding boxes obscure the subjects in ...
- 11
0
votes
0
answers
19
views
Do linear layer after GRU saved the sequence output order?
I'm dealing with the following senario:
My input has the shape of: [batch_size, input_sequence_length, input_features]
where:
input_sequence_length = 10
...
- 363
0
votes
0
answers
23
views
how to find out input data, its structure and how to achieve them on graph machine learning model?
I'm a newbie in graph machine learning and apologize if my question is silly. There is a model suggested in some paper for inductive link prediction, I need to use that model on my custom graph but I ...
- 1
1
vote
1
answer
91
views
What is the right processing order when working with a dataset that already consists of test and train data?
I want to work on the following task:
Text Classification using Deep Learning models and a Transfer Learning model.
The notebook that I'm creating should include the following steps:
Data ...
- 135
0
votes
0
answers
48
views
Effect of torch weight_norm when dim = None
torch.nn.utils.weight_norm(module, name='weight', dim=0)
When dim = None, g parameter becomes equal to $\|v\|$.
Therefore, $w=g \frac{v}{\|v\|} = v$.
So, I think ...
- 101
0
votes
2
answers
46
views
In a convolutional layer, is it standard practice to modify stride and padding to get a desired output?
I'm trying to implement the CNN described in A Framework of Hierarchical Deep Q-Network for Portfolio Management (see screenshot). In the paper, the author describes the first CNN layer as having a ...
