Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
716
questions
0
votes
0
answers
66
views
Given 20+ input numbers, predict 4-5 output numbers, using past data with the same 20+ inputs / 4-5 output for training
I'm a veteran Python software engineer, but very amateur at this stuff. I've used PyTorch for some NLP (sentiment and classification), and I've spent a bit of time attempting to learn data science, ...
- 101
0
votes
0
answers
433
views
Getting Error: TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not tuple
I am working on a CNN multi-class classification of different concentrations (10uM, 30uM, etc.) I create my dataset to include the images as the features and the concentrations as labels. Note that ...
- 23
0
votes
0
answers
144
views
is the distilRoberta transformer model overfitting or underfitting?
I am a bit new to ML, below are the results after I fine tuned distilRoBERTa using HuggingFace Trainer. I cant tell if my model is over-fitting, under-fitting or ok? I ran 7 epochs.
I think its ...
- 1
0
votes
0
answers
103
views
Multi-label classification inference
I am working on a multi-label classification with transformers. I want to assign tags to input text. First, I have trained a model multiclass and with the pipeline function I can retrieve all possible ...
- 1
2
votes
0
answers
883
views
How to implement an RBF network in Pytorch?
This is how I understand an RBF network.
From a set of points $P:=\{p_i\}_{i \in I}$ and values $F:=\{f_i\}_{i \in I}$, an RBF network is an approximation $\tilde{f}(x)$,
$$\tilde{f}(x) = \sum_{j \in ...
- 160
0
votes
0
answers
50
views
Estimate train complexity of a Pytorch model
I guess there are better "keywords" than that, especially I'm not sure about the "complexity" word. But I thought of none.
Let's say I have a NLP model with 1. a Embedding Layer 2. ...
- 323
0
votes
1
answer
372
views
Can __getitem__() in a PyTorch Dataset return a random sample?
Is __getitem()__ in a PyTorch Dataset restricted to always returning the same sample for the same index? I am thinking that the ...
- 125
0
votes
0
answers
41
views
Why are all PyTorch dataloader proccesses in S state (interruptible sleep) except one?
Short version: I'm using PyTorch dataloading library to load data in parallel for training a deep learning model. When I look at the CPU usage with htop, I see a ...
- 377
2
votes
1
answer
147
views
Ways to share Pytorch model without revealing architecture?
We are trying to give a model to collaborators but would like to protect the IP. What are some ways to encrypt/hide/compile the definition when sharing a trained model?
- 31
0
votes
0
answers
38
views
LightningDataModule with Trainer in PytorchLightning automatically fits validation model?
I try to fight with overfitting, this is why I decided to look through documentation (https://pytorch-lightning.readthedocs.io/en/stable/common/evaluation_basic.html#train-with-the-validation-loop), ...
0
votes
0
answers
84
views
Should i include model.train() with Dropout in PytorchLightning
I was reading guide in which an author used model.train() in each epoch because of the DropOut layer (he didn't use Pytorch Lightning). The question is - should i include model.train() in my Pytorch ...
0
votes
0
answers
22
views
How to improve pytorch network to be more precise
I have a pytorch neural network which is being trained on data containing multiple financial values and some other non numeric values translated to numbers with self created logic (for example if ...
- 101
0
votes
0
answers
737
views
How to deal with DataCollator and DataLoaders in Huggingface?
I have issues combining a DataLoader and DataCollator. The following code with DataCollatorWithPadding results in a ...
- 101
0
votes
0
answers
27
views
Object detection (with bounding boxes) end-to-end as an auxiliary task (as in multitask learning)?
Let's say that I have a visual encoder (CNN or ViT) which outputs a volume of local features of dimensions WxHxD plus a global feature vector of dimension D, which I'm currently using as the backbone ...
- 377
1
vote
2
answers
633
views
Will ML frameworks work with integrated graphics card in laptop?
I'm CS student. I'm just getting started in data science and machine learning. Can i use laptops with integrated graphics card in the data science and machine learning projects?
Will pytorch and ...
- 11
0
votes
0
answers
100
views
Transposed Convolution in PyTorch Visualisation
I am trying to visualize the output of the transpose convolution in pytorch to better understand the operation. Here is my code, inputs and output images as well:
...
- 37
0
votes
0
answers
41
views
Pruning using BERTology
I am trying out some BERT based models for a question and answering task. I need models trained on squad v2.0. To cut down on the inference time , I'm trying out pruning. I came across the BERTology ...
- 103
0
votes
0
answers
41
views
How to annotate a custom dataset to run Yolov7?
I have a personal dataset of mine on which I want to use the yolov7 network (pytorch). How do I manually annotate the dataset (assuming I know which part of the images I know to annotate) to train the ...
- 482
0
votes
1
answer
743
views
Requirements for variable length output in transformer
I have been working on modifying the transformer from the article The Annotated Transformer. One of the features I would like to include is the ability to pass a sequence of fixed length, and receive ...
- 1
0
votes
0
answers
31
views
Help with transitioning an existing DQN into a DRQN
Hi Data Science Stack Exchange community,
To preface this post, please let me know if I need to clarify any details to receive help and/or guidance. I am new to posting on Data Science Stack Exchange ...
- 1
0
votes
0
answers
11
views
AttributeError in pytorch
I want use https://github.com/mmbejani/TikhonovRegularizationTerm. This library includes the implementation of regularization tikhonov terms that were published from 2010 until now (2020).
I am trying ...
0
votes
0
answers
60
views
How do I implement recurrent activations for LSTM/GRU cells in Pytorch?
Although Tensorflow has simple parameters with which I can initialize the recurrent activation of a GRU or LSTM cell, Pytorch does not.
What is the best way to add recurrent activation in pytorch?
<...
- 164
0
votes
0
answers
167
views
Is this function how you would take the Mahalanobis Distance between tensors?
I made an attempt to copy an implementation of the Mahalanobis Distance from the PyTorch library. I'm not sure it is right or if it is more complicated than it needs to be.
I would like a working and ...
- 187
0
votes
1
answer
32
views
What is the best way to use three different losses on two classifiers?
Two classifiers need to be trained simultaneously, and I have three losses, as shown in the figure. Classifiers 1 and 2 will be updated by losses 1 and 2. Furthermore, loss 3 should update the two ...
0
votes
0
answers
139
views
How are fp32 weights converted to fp16 post training in PyTorch?
Can the weights of a model trained in full precision be converted to half precision post-training, with or without loss of accuracy? If so, what is the maths involved behind the conversion?
Thanks
0
votes
0
answers
78
views
How to replicate MMCV conv layer with integrated batch norm using PyTorch
Question
How do I replicate the following convolution layer from MMCV using PyTorch? I cannot find any reference in the MMCV docs on how norm_cfg works.
...
- 46
0
votes
1
answer
44
views
How to differentiate Bitmojis gender?
I have seen this Project Larry-zx's Githubproject
It is used to create a .pth file which can differenciate between a bitmojis gender.
Now I have created that .pth file but no idea how to use it, ...
0
votes
0
answers
384
views
How can I improve the accuracy of my pytorch neural network for classification of tabular data?
As a newbie in 'pytorch', I am building a neural network for classification of faulty water pumps in Tazania for this competition I am also using ax-platform for ...
user144514
1
vote
1
answer
62
views
Drawing transparent bounding boxes with Torchvision
I am using Torchvision in a Python script to draw bounding boxes and then crop images based on the bounding boxes drawn. The script works well, except that the bounding boxes obscure the subjects in ...
- 11
0
votes
0
answers
20
views
Do linear layer after GRU saved the sequence output order?
I'm dealing with the following senario:
My input has the shape of: [batch_size, input_sequence_length, input_features]
where:
input_sequence_length = 10
...
- 363
0
votes
0
answers
26
views
how to find out input data, its structure and how to achieve them on graph machine learning model?
I'm a newbie in graph machine learning and apologize if my question is silly. There is a model suggested in some paper for inductive link prediction, I need to use that model on my custom graph but I ...
- 1
1
vote
1
answer
115
views
What is the right processing order when working with a dataset that already consists of test and train data?
I want to work on the following task:
Text Classification using Deep Learning models and a Transfer Learning model.
The notebook that I'm creating should include the following steps:
Data ...
- 145
0
votes
0
answers
88
views
Effect of torch weight_norm when dim = None
torch.nn.utils.weight_norm(module, name='weight', dim=0)
When dim = None, g parameter becomes equal to $\|v\|$.
Therefore, $w=g \frac{v}{\|v\|} = v$.
So, I think ...
- 101
0
votes
2
answers
126
views
In a convolutional layer, is it standard practice to modify stride and padding to get a desired output?
I'm trying to implement the CNN described in A Framework of Hierarchical Deep Q-Network for Portfolio Management (see screenshot). In the paper, the author describes the first CNN layer as having a ...
1
vote
1
answer
2k
views
Is it possible to combine models in pytorch and pytorch geometric?
I am working on a node classification problem with graphical data. I've used PyTorch to classify nodes by simply applying a network to the individual nodes (e.g., ignoring graphical structure), and I'...
- 123
0
votes
1
answer
56
views
What will happen if we apply Gradient Ascent?
I have built a simple neural network on MNIST, but instead of moving toward the opposite direction of gradients, I moved in the same direction of it just by applying( pytorch ):
...
- 263
0
votes
0
answers
38
views
The best approach and library for time-series similarity
I have a time-series classification problem with IoT signals. The training dataset has seven target signals.
I used tsai as a fastai/torch library, and I achieved satisfying results.
However, in a ...
- 3
0
votes
0
answers
102
views
How to deal with multiple and overlapping timeseries from weather prediction/forecasts in pytorch-forecasting?
Dear Data Science community,
I want to run temporal fusion transformer supposed by google research.
Meanwhile it is part of pytorch-forecasting and I want to setup a ...
- 111
3
votes
1
answer
99
views
Solving video classification problem by taking EVA Large as backbone
I am solving a video classification problem. There are 9 classes in total. At first I took ResNet as a feature extractor, this gave me 0.74 accuracy. Then I changed ResNet to EVA (I also tried Swin), ...
- 131
0
votes
0
answers
32
views
How is model training affected after randomizing the weights of an intermediate layer of a pre-trained model?
Assuming that I have a deep learning model (let's say a ResNet) pretrained on a given dataset (let's say it is ImageNet). I load that model and randomize the weights of one of the intermediate layers, ...
3
votes
1
answer
180
views
How is padding masking considered in the Attention Head of a Transformer?
For purely educational purposes, my goal is to implement basic Transformer architecture from scratch. So far I focused on the encoder for classification tasks and assumed that all samples in a batch ...
- 171
0
votes
0
answers
7
views
training a recurrent mode to learn a transition function
I have recently started using recurrent deep learning model. I am not still very familiar how to use them properly. I used "Sequential Neural Models with Stochastic Layers" method to learn ...
- 101
0
votes
0
answers
90
views
Which loss function to use image generation?
What loss function we can for image generation task and colorful and large images?
Suppose if we have an auto-encoder for an image with size of (300,300,3), we will ...
- 263
0
votes
0
answers
15
views
How to create simple images out of labels?
I was trying to build a very simple model to create simple circle images with 5 different colors and random radius, but I did not get a good result and I just receive some noisy images. I have created ...
- 263
0
votes
0
answers
58
views
How to tune hyper parameter using *least* time (resource)? Or, how to know hyper parameter is already optimal, and the fault is elsewhere?
You know, models are large and slow to train (with limited GPUs), and bad hyper parameters can lead to terrible performance. Therefore, it would be great if I can find the good hyper parameters using ...
- 103
2
votes
2
answers
661
views
Training tricks for increasing stability in mixed precision
I would love to be able to use automatic mixed precision more extensively in my training, but I find that it is too unstable and often ends in NaNs. Are there any general tricks in training that ...
- 48
1
vote
0
answers
63
views
C Drive got filled while training my dataset over yolov5
Asked the question over a cross-validated stack but got recommendations to ask at the data science stack. So, I recently bought a GPU and tried yolov5 on my personal dataset of around 1500 images. ...
1
vote
1
answer
63
views
Using python AI mnist to recognize my picture, trained accuracy is 97.99%, but accuracy to my img is less than 20%
Questions like these are difficult to debug, but you'll hopefully find more help on datascience.stackexchange.com than here
I see the suggestion that post in Data science is also a good option
Using ...
- 111
0
votes
0
answers
152
views
Can we use MSELoss and CrossEntropyLoss alongside?
Can we apply both MSELoss and CrossEntropyLoss in a single network to predict both classification and regression in Deep Learning? Suppose that we have 4 points(regression) and 5 classes(...
- 263
0
votes
1
answer
393
views
Overfitting problem with small model
I have an encoder-decoder architecture where I have used top 3 layers of Swin Transformer and few convolutional layer. I tried different approach:
i. Training the Transformer layers as well, on doing ...
- 109