Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
716
questions
0
votes
0
answers
21
views
How can I read hdf5 files stored as 1-D array. and view them as images?
I have a large image classification dataset stored in the format .hdf5. The dataset has the labels and the images stored in the ...
- 111
0
votes
0
answers
8
views
How to selectively train a deep model based on the unavailability of a subset of the feature set
I am creating a deep learning binary classification model. Each sample in the dataset contains two mutually exclusive feature sets X and Y.
Feature set X is present in all samples; however, there are ...
0
votes
0
answers
16
views
Saving and Loading PyTorch Models for Inference without Model Definition
I'm working on a PyTorch project where I need to save and then later load a model for inference in an environment where the model definition is not available. Essentially, I want to load the model (...
0
votes
0
answers
24
views
Assign layers and weights in BERT
I print the weight names and shape of the BERT transformer. Now, I want to assign the printed weight to the layers in the transformers architecture:
In the following, I can assign query, key and ...
- 101
0
votes
0
answers
9
views
Pytorch resume model training
I've preserved the model's state_dict(), but unfortunately, the optimizer's state_dict() is lost. Can I still continue training the PyTorch model from the last checkpoint?
Please confirm any ...
- 1,252
0
votes
0
answers
11
views
How can I set two anchors with different widths and the same height using PyTorch’s AnchorGenerator module?
I’m trying to use the AnchorGenerator module in PyTorch, but I’m having trouble setting up two anchors with different widths and the same height. Specifically, I want one anchor to have a height of 43 ...
0
votes
0
answers
77
views
Neural Net not able to learn simple analytical equation
I am currently making my first attempts with Pytorch. I am trying to solve a simple equation with a neural net. Analytically solved, the result of my neural net shall look like this:
$$
y = \frac{x_5}{...
- 1
0
votes
0
answers
24
views
0
votes
0
answers
12
views
Is there any advantage to providing multi-dimensional input to torch modules?
Most layer types in torch.nn such as torch.nn.Linear accept input with more than one dimension. Is there any advantage in doing so if you can shape your data to represent a certain arrangement in ...
- 1
0
votes
0
answers
41
views
Can't overfit Transformer Encoder
In the below code I am trying to train a very simple Transformer Encoder model to basically do nothing with its input. Giving some arbitrary input vector x, the aim of the model is then to output that ...
0
votes
0
answers
23
views
Should non-trainable functions be part of a nn model?
Some explanation for the somewhat obscure title:
I want to train a model which can produce images given some input data. However, actually I want the model to learn some abstract representation which ...
- 101
0
votes
1
answer
9
views
Transforms applied by DataLoader in PyTorch
I've noticed that PyTorch's DataLoader is applying scaling to the input data with the MNIST dataset. I'm guessing its some sort of normalisation or scaling, but how do I actually find what transform ...
- 101
0
votes
0
answers
9
views
Loss MAE when estimating the angle of rotation of an object in an image is stuck at about 90
I am dealing with the problem of estimating the angle of rotation of objects in images. The problem is that the network gets stuck when training at a loss level of about 90.
Below is the code for my ...
- 1
0
votes
1
answer
43
views
PyTorch ResNet implementation's Training Loss increasing with every Epochs
I'm implementing a ResNet network from scratch using PyTorch. This network is unique to my requirements, since I need to perform Image Classification for Satellite Imagery with 14 different channels ...
0
votes
0
answers
8
views
Pretrained computer vision models that accept as input a segmented image and the original image
My data is a set of segmented images with extra details:
there is 30 object classes
each object is labeled with its state (very old, old-fashion, modern)
and each object is also labeled with a second ...
- 1
0
votes
0
answers
9
views
How do we modify the early stopping procedure to account for better losses after initial rise in losses?
I have a question regarding the usage of early stopping in the training of my forecasting model. Curious about how the training would go without early stopping, I observed that the test loss seems to ...
0
votes
0
answers
23
views
Generator loss not decreasing while training GAN
I’ve been attempting to create a basic GAN to generate images using this database of flowers (https://www.robots.ox.ac.uk/~vgg/data/flowers/102/).
I’ve spent a few days on this, and largely based my ...
0
votes
0
answers
31
views
Improving Wake-Word Detection Model Performance: Seeking Advice and Suggestions
I was assigned a task to train a wake-word detection model. Basically, it's a binary sequence classification model on audio samples where it should be 1 if it recognizes the wake word being said (e.g. ...
- 71
0
votes
0
answers
36
views
Avoid killing learnable parameters when transforming input into intervals
I'm trying to make a model using Pytorch which is training and transforming a set of coordinates, and then is downsampled using the model below. However, when I'm making the input coordinates into ...
- 1
0
votes
0
answers
15
views
Validation loss hump in LSTM
I'm using PyTorch to fit an LSTM to a binary time series dataset which has about 300 time series of about 20 items. I am using 15% of the time series as a validation set. I then have an MLP on top of ...
- 101
0
votes
0
answers
74
views
PlotNeuralNet plot interpretation
I am new to machine learning, I know how to build a simple neural network, but I don't know how to visualize my model accurately.
PlotNeuralNet is really an amazing ...
- 101
1
vote
1
answer
45
views
What're the standard ways of padding data for GNNs?
I am working on materials property prediction using GNNs with torch_geometric.
Each data in my dataset has different number of feature vectors x, edge_index vectors ...
- 11
0
votes
1
answer
76
views
Training ResNet50 model for binary classification
I want to use ResNet50 model to perform binary classification on a dataset spectrogram dataset. In order to do that I had to make a couple of modifications to the model's architecture:
Modified the ...
0
votes
0
answers
21
views
Discrepancy in the measured metrics in my segmentation models
I’ve trained my image segmentation models using SegmentationModelsPytorch. Three annotators marked up objects. All pixels that were voted as an object pixel by two annotators were marked as the pixels ...
- 201
0
votes
1
answer
60
views
how to define a linear function WX+b in pytorch?
I am practicing pytorch. I want to define a linear function Y=WX+B for inputs shape as (3,32,32) and output, the same shape i.e. (3, 32, 32). I defined m network as:
...
- 57
0
votes
0
answers
30
views
How to use Bertweet model for topic modeling
The problem is implementation of Bertweet in a topic-modeling project with understandable output like BERTopic, i want to use it on a relatively large (20k tweets) unlabelled dataset to segment it ...
- 1
1
vote
1
answer
213
views
issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory
I am trying to load the ckpt file and getting error
PytorchStreamReader failed reading zip archive: failed finding central directory
Here is the code
...
- 35
1
vote
0
answers
182
views
How to implement 2d Rotary Position Embedding in PyTorch?
The original RoPE paper suggests that the Rotary Position Embedding it describes can easily be extended to two or more dimensions: 3.2.2 in https://arxiv.org/abs/2104.09864 . I'm trying to find a ...
- 111
0
votes
1
answer
182
views
why I got TypeError: linear(): argument 'input' (position 1) must be Tensor, not int in NN?
I am writing a NN in pytorch. I have a list of tensors as input i.e. I created a list Y by appending 1000 tensor vectors(linear tensors) of size 3072. So, each Y[i] is a linear tensor of size 3072. ...
- 57
0
votes
0
answers
17
views
How to define a DataLoader or Loss for a e.g. multivariable functions?
I am trying to write a NN for estimating a f:R^n --> R^m. My problem is how to train network. I mean if I want to define a dataloader, how to attach X \in R^n to its related Y \in R^m ? Because ...
- 57
2
votes
3
answers
430
views
torch cuda not able to identify gpu on aws g4dn.xlarge
I have created an EC2 instance with GPU g4dn.xlarge and launched it. I want to run some ...
- 35
0
votes
0
answers
9
views
Deep Learning Methods for Video Classification
I'm working on a dataset with ~300 videos that last from 9 to 13-minute interviews of each subject and it has all the personality-related metadata that was collected during initial surveys. Which Deep ...
0
votes
0
answers
17
views
How to autosave a mode parameters during training neural network?
I am still new to python and NN. As NNs trainings is done for example 500 epochs, how I can auto-save the model so that if my connection gets lost or ran out of google GPU, next time I do not start ...
- 57
0
votes
0
answers
43
views
Can we show only results of some epochs in tqdm?
I am training a NN and use tqdm for showing the results. However, the bad thing is that it shows the results for every epoch. This is too many as I want to train NN for atleast 500 epochs.
Is there ...
- 57
0
votes
0
answers
17
views
Errors in results after saving model vs using directly from memory
I am trying to save a Fine Tuned model using trainer.save_model() but after I load the saved_model it just responds with the input back again and does not give any ...
- 191
0
votes
0
answers
7
views
Parametrizing a decreasing curve
I am trying to estimate a curve that by construction has to start from (0,0) and be decreasing. My current approach is to predict 20 numbers $d_i$ on [0, 10) as the differences between values on the ...
- 101
0
votes
0
answers
17
views
Expanding torch dataLoader by repeating elements (or something equivlent)
I have a pytorch classifier with stochastic elements. For each MNIST minibatch, I would like to feed it in K times (from the same starting state), and then average the resulting weight updates in ...
- 101
0
votes
2
answers
84
views
How to add multiple embeddings (layers) to LSTM layer
The similar question was asked before here https://stackoverflow.com/questions/52627739/how-to-merge-numerical-and-embedding-sequential-models-to-treat-categories-in-rn/52629902#...
0
votes
1
answer
137
views
How to Use Multiple Adapters with a Pretrained Model in Hugging Face Transformers for Inference?
I have a pretrained Llama-2 model in the models_hf directory and two fine-tuned adapters: a summarization adapter in ...
- 23
0
votes
0
answers
16
views
2 regression models giving same performance
I have 2 regression models (1 is deeplearning based) and another is SVR both trained on the embeddings obtained from the last FC layer of ResNet50. Output variables are min-max normalized to get in ...
0
votes
0
answers
55
views
Why I am getting error in dataloader in defining a NN?
I am trying to write a NN. However I am getting error.
Here is my Code:
...
- 57
0
votes
0
answers
32
views
What is wrong with my Torch HMDB51 Dataloader?
I'm trying to prepare a HMDB51 dataset for some image classification tasks. Have done this many times before in TF but this time I am working in PyTorch. Running into this strange problem when ...
- 1
0
votes
0
answers
35
views
Flickr8k+PyTorch, CNN+LSTM predicts always same words during model testing
I'm a beginner in Machine Learning and I'm working with the Flickr8k dataset (it contains ~8000 images, every image has 5 captions: ~40000 pairs).
I splitted the dataset in training (70%) and ...
- 1
0
votes
0
answers
44
views
Swin Transformer Relative Position Biases
I was reading the swin transformer paper and looking at the github implementation, i noticed that when calculating the relative position bias the input to the log function before the CPB MLP is scaled ...
- 1
0
votes
1
answer
30
views
Appropriate input size for nn.Embedding
I’m quite new to using Pytorch and deep learning. What size of unique categories of a categorical variable is appropriate for applying the nn.Embedding ideally (best practices)? for example, if a ...
0
votes
0
answers
42
views
0
votes
0
answers
71
views
Training with optuna-tuned hyperparameters leads to different results
I'm training an image classifier in Pytorch Lightning and tuning hyperparameters with Optuna. When I use the best hyperparameters to train a separate model, the accuracies differ from those obtained ...
- 33
0
votes
0
answers
21
views
How to dynamically set value while initializing model in Pytorch?
I want to set the in_features parameter in the Linear layer, but I want to dynamically set it while initializing the model. Because I will be getting that value in ...
- 193
0
votes
0
answers
39
views
Similiar reconstruction for Pytorch VAE
This is my first question here, so if I don't offer enough information for my question to be answered, please let me know.
I am currently working on my Bachelor Thesis, in which I aim to integrate ...
0
votes
0
answers
11
views
Using auxiliary softmaxes to measure impact of each submodule on the final softmax classifier
I am attempting to assess the impact of various submodules (CNN 1D, CNN 2D, CNN 3D, FFNN) on the final classifier of the neural network that i am currently building. The neural network itself is ...
- 101