Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Convolutional AE always overfitting time series - what’s wrong?

I've build a CAE for anomaly detection in time series, but it is always overfitting. I've tried data augmentation, short/long inputvector, dropout rates... I don't know what I'm doing wrong, may be ...
-1
votes
0answers
6 views

complexity ratio between forward and backward propagation

Im training a machine laerning model as you can see in the image (convolutional and fully connected layers only). batchsize = 64 optimizer= SGD What is the complexity ratio between forward and ...
1
vote
1answer
17 views

How to interpret the value of categorical cross entropy?

Categorical cross-entropy loss is usually used in settings where the target in one-hot encoded. Suppose I have a problem where there are 300 possible outcomes, and thus my final fully connected layer ...
0
votes
1answer
7 views

Target and output shape/type for binary classification using PyTorch

so I have some annotated images that I want to use to train a binary image classifier but I have been having issues creating the dataset and actually getting a test model to train. Each image is ...
0
votes
0answers
9 views

Can tensorboard be used after training has finished, to compare models?

I've been running a few experiments, on top of fairseq/pytorch, and have a separate log file for each experiment. It is just the default log format; if I grep for test results, across all model log ...
0
votes
1answer
12 views

Normalized 2D tensor values are not in range 0-1

Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
-1
votes
0answers
10 views

How to use PyTorch CrossEntropyLoss? [closed]

For CrossEntropyLoss, the PyTorch documentation says the input (output of my CNN) needs to have the shape (minibatch, C) and my target (labels) need to be a "1D tensor of size minibatch". ...
0
votes
2answers
29 views

How to read the predicted label of a Neural Netowork with Cross Entropy Loss? Pytorch

I am using a neural network to predict the quality of the Red Wine dataset, available on UCI machine Learning, using Pytorch, and Cross Entropy Loss as loss function. This is my code: ...
0
votes
0answers
11 views

Decoupling the performance impact of batch sizes from their impact on training speed

We are training a sequence-to-sequence-model to do something like Named Entity Recognition using PyTorch. It turns out, we get the best results with batch size 1. However, this slows down training ...
0
votes
0answers
34 views

How can implement this equation in code(Pytorch)

I am implementing a NLP paper. I am struck here with this equation, how to convert it to PyTorch? Sigma(h0*W1*h1 + h1*W2*h2) Here W1&W2 are weights. H0,h1 and ...
0
votes
0answers
21 views

VAE KL-divergence with non-standard mean

I know I can make a VAE do generation with a mean of 0 and std-dev of 1. I tested it with the following loss function: ...
1
vote
1answer
34 views

Transfer Learning on Resnets/VGGs — Validation accuracy can never be over 75%

I am trying to classify skin cancer images into two categories -- malignant and benign. Literatures suggest that using pre-trained resnet/vgg network achieves more than 90% accuracy. However, with my ...
0
votes
0answers
15 views

Why is torch complaining about an empty tensor?

The structure_loss method is supposed to return a loss for ground truth vs predicted masks: import numpy as np import torch import torch.nn.functional as F import ...
-1
votes
1answer
26 views

Pytorch's CrossEntropyLoss? [closed]

Can anybody explain what's going on here? I thought I knew how cross entropy loss works. I have tried with Negativeloglikelihood as well?
0
votes
1answer
17 views

How to deal with big output values after classification layer during training?

In AI libraries such as, Tensorflow, Keras etc., how the big output numbers are dealt during training process? For example, for classification layer, it will have some outputs such as ...
0
votes
2answers
42 views

PyTorch: Predicting future values with LSTM

I'm currently working on building an LSTM model to forecast time-series data using PyTorch. I used lag features to pass the previous n steps as inputs to train the network. I split the data into three ...
0
votes
1answer
24 views

How is the validation set processed in PyTorch?

Say, one uses the MNIST dataset and splits the provided training data of size 60,000 into a training set (50,000) and a validation set (10,000). The provided test data of size 10,000 is used as the ...
0
votes
0answers
19 views

Using nearest neighbour as mapper of xy coordinates

This is my first post here so I apologize if this is not right place for this kind of question. I am looking for some tips on using (k)nearest neighbor algorithm as a mapper of hypothetical position ...
0
votes
0answers
24 views

T5 finetuning using pytorch-lightning

I am trying to fine tune t5 to train the mode over a huge data set my problem is very close to this issue except I am out of RAM not gpu memory as you can see from the screen shot the number of ...
0
votes
0answers
14 views

Batching data for LSTMs vs fully connected models

I've implemented an LSTM auto-encoder. It trained well, and does what I want it to so far. But, I think I've misunderstood something fundamental about lstms. In a simple dense network whose input ...
1
vote
1answer
40 views

How to properly set up neural network training for stable accuracy and loss

I have a DenseNet121 implemented in Pytorch for image classification. For now, the training set-up is pretty straightforward: the data is loaded. An important characteristic here is that the ...
0
votes
0answers
18 views

How to get confidence score from a PyTorch based BiLSTM-CRF model

I have created a BiLSTM-CRF based NER model referring the following link: https://pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html Now, the ask is to get the confidence score for each ...
1
vote
1answer
18 views

Train a final model with the full data

I have trained a few NLP models, measured their performances and now I want to create a final model for production trained with all the data I have available. I'm working in text classification and I'...
0
votes
1answer
47 views

What I need to write in the line level of torch.vison for 21 classes?

In this code I found , line labels = torch.ones((records.shape[0],), dtype=torch.int64) ,that there is only one class and 0 in the case of Faster RCNN is reserved for the Background. What would be ...
1
vote
0answers
24 views

Pytorch CTC Loss Unexpected Behaviour

I have used the following code to test the behaviour of CTC loss. ...
0
votes
0answers
8 views

Series Through a GPU's Window To, For Each Item, Output a Prediction and Retrain?

Perhaps I'm missing something obvious but I've not run across a Keras or PyTorch example of online training and series prediction loop implemented on a GPU with these (seemingly obvious) ...
1
vote
0answers
16 views

Dying gradient issue in Graph Neural Networks

I am using Pytorch-Geometric library to implement a Graph Convolutional Layer(GCN) followed by few linear layers for a prediction task. But after training on graphs with np. of nodes being 10K and no. ...
0
votes
1answer
14 views

Get data from intermediate layers in a Pytorch model

I was trying to implement SRGAN in PyTorch and I have to write a Content loss function that required me to fetch activations from intermediate layers for both the Generated Image & Original Image. ...
0
votes
1answer
41 views

How to Connect Convolutional layer to Fully Connected layer in Pytorch while Implementing SRGAN

I was implementing the SRGAN in PyTorch but while implementing the discriminator I was confused about how to add a fully connected layer of 1024 units after the final convolutional layer My input ...
0
votes
0answers
7 views

How to run inference of 2 deep learning models simultaneously on video?

I want to get inference of 2 models. First model(Runs at 20fps, Pytorch), Second one is a heavier model(Inference time 1 sec, Tensorflow) on webcam feed. The first model would be running on every ...
0
votes
1answer
12 views

Torch tensor view and stride mechanics

I want my Mx2xN float tensor to become MxN complex tensor. In this minimal example I supply the the 10x2 matrix which should become the vector [0+10j,1+11j,2+12k,...] ...
1
vote
1answer
27 views

Optimizing regression weights for NN outputs with PyTorch

So I'm basically trying to fit a regression on the relation of the input and output of a neural network model. Then the idea is, that these estimated regression weights should be optimized to some ...
0
votes
0answers
34 views

What exactly negative/positive value of Captum's Integrated Gradient mean?

I use Captum's Integrated Gradient to interprete my PyTorch's neural network. I know that from github and original paper mentioned that ... Positive attribution score means that the input in that ...
0
votes
1answer
37 views

Why does BERT embedding increase the number of tokens?

I am new to DataScience and trying to implement BERT embedding for one of my problems. But I am having one doubt here. I am trying to embed the following sentence with BERT - "Twinkle twinkle ...
1
vote
1answer
26 views

PyTorch: LSTM for time-series failing to learn

I'm currently working on building an LSTM network to forecast time-series data using PyTorch. I tried to share all the code pieces that I thought would be helpful, but please feel free to let me know ...
0
votes
0answers
35 views

Levenberg–Marquardt or Adam optimization

In RBF (radial basis functions)-Neural Networks, which method (Levenberg–Marquardt or Adam optimization) is more efficient for optimizing the parameters (centers, widths, and output weights) in ...
1
vote
1answer
22 views

Gradient passthough in PyTorch

I need to quantize the inputs, but the method (bucketize) I need to do so is indifferentiable. I can of course detach the tensor, but then I lose the flow of gradients to earlier weights. I guess ...
1
vote
1answer
21 views

Pytorch Luong global attention: what is the shape of the alignment vector supposed to be?

I am looking at the Luong paper on Attention models and global attention. I understand how the alignment vector is computed from a dot product of the encoder hidden state and the decoder hidden state. ...
0
votes
0answers
12 views

Learning affinity among features

A batch of semantic objects in the image (lesions in CT scans) are represented in feature space, $X_{B \times C}$. I want to represent the whole batch in a single vector, $1 \times C$, in order to ...
0
votes
1answer
140 views

BERT :dropout(): argument 'input' (position 1) must be Tensor, not str

I am new to NLP and would like to build a BERT model for sentiment analysis so I am following this tutorial. However, I am getting the error below: ...
0
votes
0answers
22 views

How to use Pytorch's Transformer module “out of the box”

I am working on implementing my first transformer, and recently I've been working in Pytorch and I see that they offer a pre-packaged transformer model. Here are the docs. I have been reading through ...
0
votes
1answer
46 views

Mask R-CNN Background Subtraction Implementation

I am currently attempting to reimplement a paper on fall detection (https://ieeexplore.ieee.org/abstract/document/9186597). It requires a background subtraction algorithm called Mask R-CNN. Are there ...
0
votes
0answers
173 views

implementing skip connection in neural nets in pytorch

I'm trying to implement skip connections in neural nets for tabular data in pytorch code: ...
0
votes
0answers
7 views

MSE errors on autoencoder for dim reduction decreases in a weird patteren and I would love some help to dechyper it

I'm training a denoising autoencoder right now to reduce the dimension of a feature vector I designed of dim 58 to a latent space of dim 10, or less hopefully. I'm having a hard time understanding ...
0
votes
0answers
18 views

Saving model with pytorch

Two of the widely known ways of saving a model’s weights/parameters. torch.save(model.state_dict(), ‘weights_path_name.pth’)  It saves only the weights of the ...
0
votes
1answer
138 views

How to define the input channel of a CNN model in Pytorch?

In pytorch, we use: nn.conv2d(input_channel, output_channel, kernel_size) in order to define the convolutional layers. I understand that if the input is an image ...
0
votes
1answer
31 views

Continuous Bag Of Words (CBOW) network architecture?

Looking into word2vec like embeddings I found this exercise on PyTorch's website which prompts the reader to implement a CBOW network in PyTorch. My question is about the architecture to implement ...
1
vote
0answers
11 views

Common human-writable text format for specifying the architecture (untrained) of a neural network?

I am curious if there is a common human-writable text format for specifying the architecture (untrained) of a neural network. There are informal notations such as described here. I am aware of ONNX ...
0
votes
1answer
28 views

Why does my GPU immediately run out of memory when I try to run this code?

I am trying to write a neural network that will train on plays by Shakespeare and then write its own passages. I am using pytorch. For some reason, my GPU ...
1
vote
1answer
30 views

How to train a neural network on multiple objectives?

I have a multi-class neural network classifier that has K classes(products). For every row, only one of the classes will be 1 at a time. Now, this approach works fine if I have only 1 objective to ...

1
2 3 4 5
8