Questions tagged [pytorch]
Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.
716
questions
0
votes
1
answer
56
views
Normalising Image Data
Hi I am wondering when it comes to normalising images across each of the channels, do you use the same scaling factors that is used for training for testing set as well or separate ones.
In ...
- 103
0
votes
1
answer
331
views
How to convert Torchvision image tensor to base64 directly?
I have this code that is supposed to convert an image entry of a Torchvision dataset to a base64 string. To do that, it serializes the tensor from a Torchvision dataset to a string, modifies that ...
- 111
2
votes
1
answer
300
views
Pytorch Neural Network that tries to approximate $z_i = x_i^2 + y_i^2$ not converging to solution
Background
I am teaching myself Pytorch, as a Mechanical engineering technology (MET) faculty. My end goal is to replace many data-driven heat transfer and Fluid dynamics models with Neural network ...
- 123
1
vote
0
answers
23
views
What is purpose of diffusion matrix? What is its role in graph clustering? How do it serve its purpose by equation?
As titled, I am not quite sure what is purpose of Diffusion Matrix, which is support to clustering graph.
From this article (https://arxiv.org/pdf/2006.05582v1.pdf) I am might able to compute the ...
- 11
1
vote
2
answers
26
views
High loss and high accuracy on validation dataset at the early stage of training binary classifier
I am training a ResNet50 network with simulation data and my validation dataset is the experimental data. The simulation data is not a 100% accurate representation of the experimental data. The ...
- 113
0
votes
0
answers
298
views
My model is not learning
I am using the ogb molhiv dataset for graph classification, I imported the data and created the DataLoader following the ogb documentation. The data is composed of 41127 graphs and there are 2 classes....
- 3
0
votes
0
answers
36
views
Input size vs hidden state in RNNs
Im using PyTorch to implement RNNs on univariate time series data. This is the documentation for the RNN class: link
I think I'm understanding the math behind an RNN cell. But I have an specific ...
- 101
0
votes
1
answer
333
views
Linear Regression in Pytorch-vanishing gradient with Softmax
I am implementing a non-linear regression using neural networks with one single layer in Pytorch. However, using an activation function as ReLu or Softmax, the loss gets stuck, the value does not ...
- 101
1
vote
1
answer
223
views
Bounding box regression without a classification task
I am using PyTorch to create a model that detects certain objects in an image. I framed my problem as a regression on bounding boxes, without any classification task whatsoever. The reasoning behind ...
- 11
0
votes
0
answers
618
views
Help with easyocr fine tuned model inference
I have trained a custom model, I have the yaml file and pth file and the py file in the correct directories. But now I face this error
...
0
votes
0
answers
2k
views
How to correctly create a PyTorch Tensor from a Pandas DataFrame?
I have loaded my data into a Pandas DataFrame, and performed some pre-processing, and then I need to convert it into a PyTorch Tensor for training as my features data.
Obviously, This new tensor do ...
- 101
0
votes
1
answer
77
views
How to implement a customized DataLoader that inherits pytorch's one?
I need to implement a customized DataLoader, that inherits from torch.data.utils.DataLoader.
I have searched it for half hour, but there is no example or doc about this.
What methods of it should I ...
- 101
1
vote
1
answer
72
views
Clustering on Market-1501 dataset
I am trying to perform clustering on the Market-1501 dataset. The approach that I am using is as follows:
I train a Person-Reid Model (using this repository: Reid-Strong-Baseline)
Use a version of ...
- 13
1
vote
0
answers
326
views
Dataset Format for fine tuning deepset/roberta-base-squad2 hugging face transformer model
I have been trying to fine tune the roberta model for QnA to my specific domain (healthcare).
I am unable to find the correct way to provide the dataset format to the tokenizer in order to fine tune ...
- 111
0
votes
1
answer
318
views
pytorchs LSTMs use of 'bias' and 'weight' strings
Hi I am new to RNN and have come across this the following implementation of Pytorchs LSTM, but I cant understand how (or why) the 'bias' and ...
- 125
1
vote
3
answers
148
views
Using the results of clustering to retrain a neural network
I am following and expanding upon previous work from the winner of the Melanoma Classification from here.
The dataset has 9 classes. The competition is only interested in the one class (Melanoma).
I ...
- 198
0
votes
2
answers
39
views
Difficulty loading data/running model on custom dataset derrived from DNA sequence data - TypeError when attempting to run model
I am a student who has some limited experience with keras, and for a new project recently decided to learn how to use pytorch to implement my models. I'm a beginner with both, so apologies in advance ...
0
votes
1
answer
280
views
Running Model on both GPUs and CPUs
I have access to a hpc node, of 3 GPU and maximum of 38 CPU. I have a transformer model which I run of a single GPU at the moment, I want to utilize all the GPUs and CPUs.
I have seen couple of ...
- 17
0
votes
1
answer
20
views
What says the output of autoencoder?
What is the meaning of output of autuencoders? Can we say it is the noise removed version of actual dataset and should it be symmetrical?
- 11
2
votes
0
answers
119
views
how Can we add extra word embedding to the pytorch funnel transformer?
i was approaching NLP sequence classification problem (3 classes) using huggingface transformers (funnel-transformer/large) and tensorflow.
first i created laserembedding like this :
...
- 121
0
votes
1
answer
122
views
Using 'Mlxtend' with 'TensorFlow' or 'Pytorch'
Is it possible to create a simple stacking implementation for regression with 'Mlxtend' using models created by 'TensorFlow' or 'Pytorch' however the documentation only supports examples that contain '...
- 101
0
votes
1
answer
642
views
What is hidden in torch.nn that is multiplied by the feature tensor of my data?
Redoing a tutorial on Captum I have recreated its neural network, TitanicSimpleNNNModel, a simple architecture whereby the last layer performs a softmax operation and has 2 units, corresponding to the ...
1
vote
1
answer
287
views
Difference between pretrained, finetune, feature extract
I'm a little confused between the following terminology: pretrained, finetune and feature extract. I would like to use an out-of-the-box model to train a covid dataset. If I were to use resnet, would ...
- 85
0
votes
0
answers
34
views
Why is there a 0.5 in this loss?
I'm reading this paper and I don't understand why the squared L2-norm is also multiplied by 0.5 in the loss.
They want a loss that measures the distance between two feature maps. Why don't they use ...
1
vote
1
answer
5k
views
Using Dataloader to display an image
I have successfully loaded my data into DataLoader with the code below:
train_loader = torch.utils.data.DataLoader(train_dataset, 32, shuffle=True)
I am trying to ...
- 85
1
vote
1
answer
57
views
Fine tuning Convolutional Neural Network with a learnable first layer
I have a classification task using grayscale images and I want to leverage from pretrained networks.
There are a lot of resources out there presenting how to fine tune large neural nets like resnet, ...
- 65
2
votes
1
answer
4k
views
Found input variables with inconsistent numbers of samples: [908, 9080]
I have a dataset, I have reconfigured my tensors as a single 3072 sized line array. I have reconfigured the valid dataset and training dataset. You can find all of the information about my train, ...
- 21
0
votes
0
answers
85
views
training loss decreases but training accuracy is also decreasing with epochs
I am working on the classification problem where by I am having a hinge loss function + other loss terms to optimize for which the input is the output from tanh layer at the end. But I can't reveal ...
- 121
0
votes
0
answers
132
views
Pytorch CNN in_channels, out_channels for Classifying DNA Sequences
Apologies as this question has been asked before -- I'm really trying to wrap my head around the motivation behind designing neural network architecture.
I'm designing a convolutional neural network ...
- 1
1
vote
1
answer
53
views
Understand the reason of embedding and the size inside it in Pytorch
I'm very new to pytorch - taking a course in udemy.
There is something I find hard to understand and would like to get explaination about, in a bit simpler words than what I can find in the ...
- 349
1
vote
1
answer
113
views
In LSTM why h_t output twice?
According the LSTM design:
The hidden state (ht) is output twice (1 and 2 in the picture).
If they are the same, why we need them twice ?
Is there a different use for each one of them ?
According to
...
- 363
0
votes
1
answer
326
views
How to reduce the size of Bert model(checkpoint/model_state.bin) using pytorch
I used torch.quantization.quantize_dynamic to reduce the model size but it is reducing my prediction Accuracy score.
I'm using that model file inside the Flask and ...
0
votes
1
answer
95
views
What is the right Pytorch RNN implementation?
I read about RNN in pytorch:
RNN — PyTorch 1.12 documentation.
According to the document the RNN run the following function:
I looked on another RNN example (from pytorch tutorial):
NLP FROM SCRATCH: ...
- 363
0
votes
0
answers
28
views
Are there examples of quantization aware neural networks
I'm looking for examples of Machine Learning / Neural Networks examples that work with quantized weights, activation functions,.... The simple approach of training with floating point parameters and ...
- 101
1
vote
1
answer
1k
views
BertTokenizer Loading Problem
I loaded this BertTokenizer previously, but now it is showing, I have to make sure I don't have a local directory. In my kaggle kernel, I don't have this local directory.
How to solve it?
...
- 13
2
votes
0
answers
2k
views
Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same
I am trying to run this notebook on Apple M1 (1st gen) running MacOS 12.4,
libs freeze:
...
- 121
0
votes
0
answers
38
views
extract features from parts of one image
I have several parts of one image that have one caption... I need to do image captioning by evaluating every part of the image to which the caption will belong so do I need to extract the features ...
- 1
0
votes
0
answers
464
views
Neural Network Stuck at Low Accuracy
I am new to deep learning so forgive me if this is an obvious mistake, I have tried to find similar questions online yet none seem relevant to my problem.
I am using pytorch for image classification ...
0
votes
2
answers
438
views
Pretrain RoBERTa model with new data using PyTorch library
I've pretrained the RoBERTa model with new data using a 'simpletransformers' library:
...
- 125
1
vote
1
answer
2k
views
What is the purpose of Sequence Length parameter in RNN (specifically on PyTorch)?
I am trying to understand RNN. I got a good sense of how it works on theory. But then on PyTorch you have two extra dimensions to your input data: batch size (number of batches) and sequence length. ...
- 13
1
vote
0
answers
37
views
Accuracy drops when adding a fully connected layer for dimensionality reduction to a ResNet50
I'm training a ResNet50 for image classification and I'm interested in decreasing the dimensionality of the embedded layer, in order to apply some clustering techniques.
The suggested dimension is ...
- 61
0
votes
1
answer
24
views
expanding 1x1xn global average pooling output to the size of the input map HxWxn
I am trying to recreate the model in this paper : BiSeNetV2
There is one module called Context Embedding Block, in which Global Average Pooling is used to embed global information into the feature ...
- 31
0
votes
1
answer
207
views
How do i generate text from ids in Torchtext's sentencepiece_numericalizer?
The torchtext sentencepiece_numericalizer() outputs a generator with indices SentencePiece model corresponding to token in the input sentence. From the generator, I ...
- 17
0
votes
1
answer
2k
views
How to make an ensemble model for classification with pytorch using trained models?
I am trying to make an ensemble model composed of two pre-trained models, using torch, in order to classify an image.
Below is some code, based on this post.
...
- 123
0
votes
0
answers
65
views
From what function do come the gradients that I use to adjust weights?
I have a question about the loss function and the gradient.
So I'm following the fastai (https://github.com/fastai/fastbook) course and at the end of 4th chapter, I got myself wondering.
From what ...
2
votes
0
answers
251
views
How to train a Task Specific Knowledge Distillation model using Hugging face model
I was referring to this code:
https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb
From @philschmid
I could follow most of the ...
- 277
1
vote
0
answers
276
views
How does bert produce variable output shape?
Suppose if I provide a list of sentences:
['I like python',
'I am learning python', # longest sentence of length 4 tokens
'Python is simple']
Bert will produce ...
- 11
1
vote
0
answers
1k
views
How to save hugging face fine tuned model using pytorch and distributed training
I am fine tuning masked language model from XLM Roberta large on google machine specs.
When I copy the model using gsutil and subprocess from container to GCP ...
- 277
0
votes
0
answers
613
views
Do the multiple heads in Multi head attention actually lead to more parameters or different outputs?
I am trying to understand Transformers. While I understand the concept of the encoder-decoder structure and the idea behind self-attention what I am stuck at is the "multi head part" of the &...
0
votes
0
answers
1k
views
Hugging face Model Output 'last_hidden_state'
I am using the Huggingface BERTModel, The model gives Seq2SeqModelOutput as output. The output contains the past hidden states and the last hidden state.
These are my questions
What is the use of the ...
- 17