Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
723 questions
0
votes
0
answers
4
views
How do I create mini-batching to meet my training requirements?
I am working on timeseries dataset. There are 13 timeseries. First 10 of them are actually input features and last 3 are ground truth targets that model needs to learn to predict. I am working with ...
- 105
1
vote
0
answers
11
views
How to solve the issue with getting free ports in Pytorch DDP?
I am facing issues with getting a free port in the DDP setup block of PyTorch for parallelizing my deep learning training job across multiple GPUs on a Linux HPC cluster.
I am trying to submit a deep ...
3
votes
1
answer
32
views
RNN performing worse than random guessing on large dataset
I have to start off by saying I am 100% a beginner here.
I trained a RNN model on a 30 class dataset with over 90000 samples and it achieved less than 2% accuracy. Training the same model on a small ...
- 31
1
vote
0
answers
16
views
How to Visualize a PyTorch Model Without Input Parameters?
I’m working on a project that heavily relies on computation graph manipulations but isn’t directly in the field of machine learning. However, we are using PyTorch due to its flexibility and support ...
0
votes
0
answers
17
views
Using ResNet50 with SE block on imbalanced data - pytorch
I worked with a breast cancer ultrasound image dataset containing 432 benign cases, 210 malignant cases, and 133 normal cases. Initially, I used a pretrained ResNet-50 model, which yielded the ...
0
votes
0
answers
20
views
Loss is not decreasing
I'm building a project on text generation using transformer architecture. I've used Huggingface tokenizer for tokenization, my dataset is daily-dialog, and my model architecture is:
...
0
votes
0
answers
16
views
How to get meaningful results from EncoderDecoder network for time series forecasting
I'm trying to traing an EncoderDecoder network for a multivariate time series input and a univariate time series output. In particular my dataset is composed of inputs of 32 features x 600 seconds and ...
- 101
0
votes
1
answer
23
views
Why does tanh activation work better with Pytorch than with Keras?
I'm doing a neural network to recognize written Cyrillic letters, and I found out that, when I use tanh activation function, it works WAY better with PyTorch than with Keras.
Keras code:
...
- 1
0
votes
0
answers
8
views
Can we add additional layers on top of a quantized model to fine tune?
Is it possible to use a quantised model (e.g. int8) and add layers (e.g. PyTorch Linear) to do fine tuning in PyTorch?
If possible, how to convert the quantized model output vectors to feed to the ...
- 791
2
votes
0
answers
8
views
What may not work in PyTorch when using qantized models?
I suppose not all the PyTorch tools and modules work with quantized (e.g. to int8) models. But what may not work and why?
Device and Operator Support
Quantization support is restricted to a subset ...
- 791
0
votes
0
answers
45
views
Why model training became extremely slow in PyTorch
I am encountering a strange behavior in the use of pytorch/cuda for training models: I noticed that training models that previously require little time to complete each training epoch, now require ...
- 101
0
votes
1
answer
43
views
Unable to fit ANN to "simple" dataset?
I am really struggling of modeling a dataset that I acquired by doing experiments. Concretely, those are time series (online) data of measurements and the objectives are kinetic parameters that I ...
- 1
0
votes
0
answers
14
views
Loss function for multi label classification with multiple digits in mnist
As the title suggests, i'm looking for a loss function to apply to a modified mnist dataset which has multiple digits. I need to predict all the digits in the image. Each image has 1-3 digits and each ...
1
vote
1
answer
34
views
Transformer spams the most frequent character
I noticed that the transformer tends to be optimized to just generate the most frequently appeared character.
For example, I have the following input tokens: ...
- 33
0
votes
0
answers
19
views
tensorflow to pytorch weight transfer
There is a modified efficientnet TF model, that I'm trying to simulate in pytorch. I have made the architecture changes to the model in pytorch, dumped the TF model weights, and loaded them back in ...
- 101
0
votes
0
answers
17
views
How to finetune time series transformer hyper parameters to beat the LSTM performance?
I am trying to train an ML model on time series data. The input is 10 timeseries which are essentially a sensor data. The output is another set of three time series. I feed the model with the window ...
- 219
0
votes
0
answers
7
views
Xavier initialization std is not as expected
The expected standard deviation of Xavier initialization of a D-dimensional tensor would be $1 / \sqrt (D)$ but it is not as below. Please help understand why.
...
- 791
0
votes
0
answers
15
views
My network to classify dialects is not working
I have written the following code to classify dialects based on the timit dataset using .wav files. For some reason my model is not learning and classifies everything into the same class. Is it ...
0
votes
0
answers
15
views
How to avoid in-place operations in a PyTorch loop to allow backpropagation?
I'm working on a PyTorch model that involves a double loop for calculations. The problem is that I'm getting an error related to in-place operations when I try to perform backpropagation.
Here's a ...
0
votes
1
answer
87
views
Train the transformer with PPO
Context
I'm trying to apply reinforcement learning to the transformer.
I have the following tokens: ...
- 33
0
votes
1
answer
32
views
Understanding the output of Google's wide and deep model?
I'm trying to implement Google's wide and deep model and I have a question about its output.
According to the equation (3) in the paper:
$$ P(Y=1|X) = \sigma(w_{wide}^T[x,\phi(x)] + w_{deep}^T a^{(l_f)...
- 103
0
votes
1
answer
16
views
Empty columns in confusion matrix
I am training on pre-processed APTOS 2019 dataset for disease grading and the last two columns of my confusion matrix are constantly zero every time. Data is distributed as class0-1805 images, class1-...
0
votes
1
answer
34
views
Transformer is not getting better
I implemented a method fit to train the model that uses nn.Transformer with the teacher-forcing approach.
Unfortunately, I noticed that the loss was not getting ...
- 33
0
votes
0
answers
14
views
Understanding the `model.fit` function in keras and imbalanced datasets
As an exercise, I'm trying to translate a model written in Keras (https://github.com/CVxTz/ECG_Heartbeat_Classification/blob/master/code/baseline_mitbih.py) into Pytorch code. I realize in Keras much ...
0
votes
1
answer
41
views
Word embedding is not getting better
I created a simple neural network to train the word embeddings.
I have 6 tokens only: ["apple", "banana", "lime", "red", "yellow", "green"].
...
- 33
0
votes
0
answers
35
views
create a python AI model that can read PCB layout image
can someone please help me create this project
i need to understand how to start it what do i need and the steps.
i never created AI model before.
** create AI model that can read PCB layout image (...
- 1
0
votes
2
answers
36
views
Pytorch `DataSet.__getitem__()` called with `index` bigger than `__len__()`
I have following torch dataset (I have replaced actual code to read data from files with random number generation to make it minimal reproducible):
...
- 219
0
votes
1
answer
21
views
Handle text column with PyTorch
I'm new in ML so question may be stupid.
I have a data set with multiple numeric columns and one text column. Text is just one sentense.
So i want to use all data avaible for classification. But i don'...
- 101
1
vote
1
answer
35
views
PyTorch noise estimator model not learning - converges to same output regardless of input
I'm implementing a simple noise estimator using PyTorch. The model consists of a few convolutional layers followed by a small fully connected network, ending with a single neuron that outputs the ...
0
votes
0
answers
25
views
How to Prepare Data for U-Net Model Training with .tif Images
I'm new to image segmentation and trying to train a U-Net model. I have a dataset consisting of .tif satellite images and their corresponding annotations. Here is a sample of my data:
...
- 1
0
votes
1
answer
27
views
Where should I learn pytorch from?
I'm a undergraduate student. I've coded a three-node neural network (that works) based on my professor's guidance. However, I'd like to pursue a career in AI and Data Science, and I'd like to teach ...
0
votes
1
answer
26
views
Why is there an extra convolution layer in VGG19 for some neural style transfer implementations?
I've been exploring neural style transfer and noticed that some implementations, such as the official PyTorch versions of Universal Style Transfer via Feature Transforms and unofficial Arbitrary Style ...
0
votes
0
answers
10
views
Methods to Interpret Physical Relationships Between Inputs and Output in a Trained ANN Model
I have trained an artificial neural network (ANN) model using PyTorch that accurately predicts a scalar output based on three input tensors with the following shapes:
Input 1: (3600, 3)
Input 2: (...
0
votes
0
answers
117
views
Recommendation System: Two-Tower Model Underperforming Simple Embedding Average Baseline
I'm trying to build a recommendation on a dataset of product purchases. The dataset consists of roughly 4 Amazon products that a particular user has bought (in sequence). I want to use the first 3 ...
- 1
1
vote
1
answer
40
views
MAPIE and pytorch
I am trying to get MAPIE conformal intervals (specifically enbpi) to work with a pytorch built FFN, that I want to use for time series modeling. The problem I am having is that I do not know how to ...
- 136
0
votes
0
answers
7
views
Vocal Tract Length Perturbation (VTLP) code
I'm looking for an example of code that uses the VLP (Vocal Tract
Length Perturbation) technique to improve speech recognition.
Regards
- 1
0
votes
1
answer
45
views
Gradient output through custom loss function
I’m very new to Pytorch (and ML in general), so I’m having difficulty understanding what is going on WRT a custom loss/cost function I’m looking at. I understand what’s going on in the function, but I ...
0
votes
0
answers
28
views
GPU running out of memory for ConvLSTM model in Pytorch
I'm trying to replicate the bidirectional convolutional LSTM proposed in Xiong et al. 2017 to predict crowd count density maps, but I'm running into memory issues during the training. This is what I'...
- 11
0
votes
2
answers
26
views
LR not decaying for pytorch AdamW even after hundreds of epochs
I have following code using AdamW optimizer from pytorch:
optimizer = AdamW(params=self.model.parameters(), lr=0.00005)
I tried ...
- 105
0
votes
0
answers
67
views
Implementing pytorch temporal fusion transformer on time series
I am trying to run the temporal fusion transformer from the pytorch package. I am trying to compare the output on like terms to the tensorflow output in this paper p. 15 https://arxiv.org/pdf/1912....
2
votes
1
answer
95
views
Level of confidence for binary classification
I’m relatively new to PyTorch and deep learning. I was able to create a model and analyze a data set for both a training and test set in a binary classification problem. Everything is working well. ...
- 21
0
votes
0
answers
41
views
Mobilenetv2 transfer learning
Goal:
Transfer learning Mobilenetv2 (input size 224x224 and it's own preprocessing (resize + central_crop + normalization)) as encoder for Unet with input size 512x512 using pytorch.
What I've done:
...
0
votes
0
answers
28
views
Cross entropy loss for multi classification problem
I am handling a multi-class classification problem, with label in the following form [1333201000]
and the logit output of the model is in the form
([[ 0.4523, 0.0198, -0.1911, -0.0036],
[ 0.4917, 0....
- 1
0
votes
1
answer
368
views
RuntimeError: Boolean value of Tensor with more than one value is ambiguous
I am trying to get boolean outputs {0,1} for my neural network. My final output is a real value e.g. r and I wanted it to be 0 if r <= 0 and be 1 if r >1. To do this, I did the following code in ...
- 103
2
votes
0
answers
26
views
Stuck with constant loss and network not learning
I am trying to predict certain function coefficients (output: a, b) based on its curve (input: frequency_response) with the help of https://github.com/Blealtan/efficient-kan (Kolmogorov-Arnold Network)...
- 151
0
votes
0
answers
11
views
How to handle sequences with crossEntropyLoss
fist of all i am ne wto the whole thing, so sorry if this is superdumb.
I'm currently training a Transformer model for a sequence classification task using CrossEntropyLoss. My input tensor has the ...
- 101
0
votes
0
answers
17
views
Unable to reproduce PyTorch model training performance
I have trained a RegNet model on a custom dataset for an image classification task. That was in August 2023. Now I want to train exactly the same model again, using the same dataset. I would expect ...
- 101
0
votes
0
answers
30
views
What is the most accurate way of computing the evaluation time of a neural network model?
I am training some neural networks in pytorch to use as an embedded surrogate model. Since I am testing various architectures, I want to compare the accuracy of each one, but I am also interested in ...
- 1
0
votes
1
answer
130
views
Why is my Transformer model outputting the same class for every token and not improving despite decreasing loss?
I'm currently training a Transformer model for a sequence classification task using CrossEntropyLoss. My input tensor has the shape (batch_size, classes, seq_len) and my target tensor has the shape (...
- 101
0
votes
1
answer
151
views
Interpretation of PPO learning curve, value loss, policy loss
my PPO training for a custom gymnasium environment resulted in following outcome. I would need some advice how to interpret the results and where to start activities to improve.
Thank you very much ...
- 71