Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Multi Input Network MNIST-CIFAR10

I have the following task of meta learning: We want that our neural network learns to sum weights. 1)Do the training on MNIST, and on CIFAR10 (as support dataset). We want that performance (accuracy) ...
1
vote
0answers
11 views

A simple attention based text prediction model from scratch using pytorch

I first asked this question in codereview SE but a user recommended to post this here instead. I have created a simple self attention based text prediction model using pytorch. The attention formula ...
0
votes
0answers
7 views

Getting gradient for gradCam in pytorch

I am using forward and backward hook in my pytorch densenet121 model. I set requires_grad to False at the time of training. ...
0
votes
1answer
16 views

Best setup for Windows and Ubuntu on same machine? [closed]

I have been using Windows 10 on my machine. However, for many deep learning applications Linux is recommended if not required. For this reason, I want to keep my Windows 10 installation but also have ...
0
votes
1answer
21 views

Numerically stable hyperbolic tangent

The hyperbolic tangent is commonly used as an activation function: $$ tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}} $$ Although, it is unclear how this function is implemented to be numerically stable ...
0
votes
2answers
21 views

Loading pretrained model with Pytorch

I saved my model with this code: from google.colab import files torch.save(net, 'model.pth') # download checkpoint file files.download('model.pth') Then uploaded ...
0
votes
0answers
18 views

Testing trained model on the image from the test set

I trained my EfficientNet (CNN) and got accuracy=0.73. The question is how to check it on one concrete image from the testing set? How to write a code in python for it? I described the testing set ...
0
votes
1answer
23 views

Adding Validation PyTorch

First of all, I'm new in this field and it's my first this kind of work. I'm trying to train EfficientNet (CNN), the code below is working fine, but I can't succeed to add also validation set to the ...
1
vote
0answers
25 views

How to interpret weird loss/accuracy behavior in cross validation

I'm using transfer learning to classify binary medical MRI brain images. I used 5-fold cross validation, SGD optimizer and scheduler with different learning rate with ...
0
votes
0answers
15 views

How to design optimizer for combined model training in Pytorch

I am trying to train a embedder. So, I have an architecture for the model to embed texts. And I have another model architecture that will take the inputs from the output of the first model and predict ...
0
votes
1answer
22 views

why my pytorch liner regression failed?

I am new to pytorch, i want start from a simple example-linear regression: I created some random training and test sample. here is my code: ...
0
votes
1answer
15 views

Which output of a BiLSTM layer should be used for classification

I am trying to implement a BiLSTM layer for a text classification problem and using PyTorch for this. ...
0
votes
1answer
10 views

Concat two tensors of different dimensions

I have two tensors. For example - a = torch.randn((500, 200, 10)) b = torch.randn((500, 5)) I have to concat each of b tensor ...
0
votes
0answers
17 views

What kind of Neural Network should I build to classify each instance of a time series sequence?

Let's say, I have the time-series dataset below-left. I would like to train a model in such a way that if I feed the model with an input like below-right, it should be able to classify each sample ...
0
votes
1answer
9 views

The learning_rate in TensorFlow code is the sum learning rate of a batch or the learning rate of a data?

import tensorflow as tf batch_size = 10 learning_rate = 0.001 tf.train.AdamOptimizer(learning_rate) So the question is: The ...
0
votes
1answer
10 views

Memory issue when trying to initiate zero tensor with pytorch

I am facing a memory issue while trying to initialize a torch.zeros - torch.zeros((2000,2000,3200), device=device) Getting the following error: ...
0
votes
0answers
4 views

Apollo optimizer memory leak in pytorch

There is a few dense layers, 2-layer LSTM network and to Linear Layers in the end. Even after I made a network very small, all GPU memory (8 GB) got consumed in a few epochs. I understand that Apollo ...
0
votes
1answer
18 views

How do I get my Neural network to ignore certain values?

I was wondering if there was a way that I can get my CNN encoder-decoder neural network to completely ignore certain values in my data (2d images). There are some pixel values of 0 that never change ...
0
votes
0answers
3 views

Train only Region Proposal Network in faster RCNN architecture

I am looking for a way to used my pretrained EfficientNetv2 model and turn it into an object detection. Is there anyway, I can put my pretrained model as a backbone and only train the region proposal ...
1
vote
0answers
39 views

Which One is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?

Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch. ...
0
votes
0answers
12 views

Vgg16 model validation accuracy is stuck

I am working on a CNN model for MRI brain images classification (Alzheimer disease), I use transfer learning method for image classification - vgg16 model trained on ImageNet (1000 classes). I’ve ...
0
votes
1answer
31 views

Pytorch: understanding the purpose of each argument in the forward function of nn.TransformerDecoder

According to https://pytorch.org/docs/stable/generated/torch.nn.TransformerDecoder.html, the forward function of nn.TransformerDecoder contemplates the following arguments: tgt – the sequence to the ...
0
votes
1answer
30 views

Combining textual and numeric features into pre-trained Transformer BERT

I have a dataset with 3 columns: Text Meta-data (intending to extract features from it, then use those i.e., numerical features) Target label Question 1: How can I use a pre-trained BERT instance on ...
2
votes
1answer
92 views

Training network 10 time slower on 16core vs 8core C++ API

Pytorch seems to run 10 times slower on a 16 core machine vs 8 core machine. Any thoughts on why that is and what/if any thing I can do to speed up the 16 core machine? Thank you Below is a list of ...
1
vote
1answer
30 views

Neural network stock market predictions in further future (>1day) approach an exponential behavior

I want to predict the trend of a specific stock using neural networks in PyTorch. I followed a guide¹ to learn about the basic structures of a program of that type. This guide, however, only works on ...
1
vote
1answer
33 views

Is time series forecasting possible with a transformer?

For my bachelor project I've been tasked with making a transformer that can forecast time series data, specifically powergrid data. I need to take a univariate time series of length N, that can then ...
0
votes
1answer
19 views

How to pass a sequence of 4 images into LSTM and CNN-LSTM

I got an assignment and stuck with it while going down the rabbithole of learning PyTorch, LSTM and cnn. Provided the well known MNIST library I take combinations of 4 numbers and per combination it ...
0
votes
1answer
19 views

Plot a training/validation curve in Pytorch Training [closed]

I have the following training method and I'm confused how may I modify the code to plot a training and validation curve history graph with matplotlib ...
0
votes
0answers
17 views

Trying to extend this code to include additional feature volume (in addition to adj close) RNN to predict adj close

I read this article on medium https://medium.com/swlh/a-technical-guide-on-rnn-lstm-gru-for-stock-price-prediction-bce2f7f30346 prep ...
0
votes
1answer
17 views

Performance metrics changing significantly based on batch size

I am working on a binary classification problem where there is significant class imbalance (minority class makes up nearly 10%). The dataset has ~15,000 observations and I have split this in to a ...
2
votes
1answer
59 views

Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.). For the purposes of this question, you may assume that I already have the input part ...
3
votes
1answer
27 views

Negative learning implementation in pytorch

I have read a paper on Negative Learning: https://arxiv.org/abs/1908.07387. The idea is that you can train a network not only by telling what label of the sample is, but by telling what it surely is ...
0
votes
0answers
21 views

How to classify all words in a sentence with a context?

I have the names of the companies (in Russian). The name can contain abbreviations, words with capital letters, words with lowercase letters, and mixed words. The model is trained according to the ...
0
votes
1answer
20 views

Update of mean and variance of weights

I'm trying to understand the Bayes by Backprop algorithm from the paper Weight Uncertainty in Neural Networks, the idea is to make a NN in which each weight has it's own probability distribution. I ...
0
votes
0answers
6 views

How to minimalize noise from autoencoder output in audio reconstruction

I am training an autoencoder that takes an audio sample and outputs a variation of the input. The network is working as expected by the final output contains noise and it's not very clear. I am ...
0
votes
0answers
18 views

How metric learning works for content based item retrieval

I was doing some computer vision experiments and recently I have started learning about metric learning and the image retrieval problem. I was experimenting with the inshop image retrieval dataset to ...
0
votes
0answers
28 views

Pytorch: how to pass the hidden state between the samples in LSTM?

I am trying to boost the performance of a object detection task with sequential information, using ConvLSTM. A typical ConvLSTM model takes a 5D tensor with shape ...
0
votes
1answer
13 views

Should I report loss on the last batch or the entire training dataset?

In the training loop below, you can see that train_loss represents the loss on the most recent batch. Whereas the eval_loss is ...
1
vote
1answer
38 views

Two-tower net does not learn when made deep

I have been trying to train a relatively simple two-tower net for recommendation. I am using PyTorch and the implementation is the following - basically embeddings layers for users and items, optional ...
0
votes
0answers
14 views

Can't get a regression problem to converge

I am working on implementing a really simple version of YOLO to learn about pytorch and building deep learning models. My dataset consists of images which have two MNIST digits placed somewhere on the ...
0
votes
1answer
12 views

GPU shows 0 utilization even when tensors and model are mounted on the gpu?

I am trying to run some PyTorch scripts on a remote GPU server. While calling the script in the ubuntu terminal i start as:...
0
votes
1answer
18 views

Understanding PyTorch's BCE Notation

According to the PyTorch documentation for the Binary Cross Entropy Loss, we can write it as follows: $$l_{n} = -w_{n}\cdot \left[y_{n}\cdot \log \left(x_{n}\right) + \left( 1-y_{n}\right)\cdot \log\...
0
votes
0answers
17 views

CNN Model Seems To Just Be Guessing

I am working with a binary classification problem, and regardless of what changes I make, the model seems to just be guessing between 0 (Negative) and 1 (Positive). The dataset is imbalanced at a ...
0
votes
0answers
9 views

Pytorch RNN with no nonlinearity

Is it possible to implement an RNN layer with no nonlinearity in Pytorch like in Keras where one can set the activation to linear? By removing the nonlinearlity, I want to implement a first-order ...
2
votes
0answers
24 views

Methods to visualize the filters in the later layers of a CNN?

I've extracted the weights from the filters of a pretrained model (AlexNet). I wish to represent these weights visually, this works fine for the first layer as there is only 3 input channels so I can ...
0
votes
0answers
15 views

Working with mySQL and pytorch Dataset

I am working with high frequency time indexed data. We have 2 types of data each with about 5 columns. For each type of data we have 2500 streams coming in and being updated every 1ms-100ms (the ...
0
votes
0answers
10 views

PyTorch LSTM with varying time steps

Is it possible to create an LSTM in PyTorch where the time steps are varying? For example, heights where measurements are taken at various times. The data might look like this: Person id Inches tall ...
0
votes
0answers
18 views

How can I create .nii (nifti) file from 3D Numpy array

I have a prediction numpy array. How can I make a .nii or .nii.gz mask file from the array?
3
votes
0answers
11 views

Running DenseNet from cmd line and jupyter but vastly Different loss and accuracy [closed]

I’m running a DenseNet121 from Pytorch with the same exact code, same exact hyper parameters and same exact image sizes, once from a jupyter notebook and once using the command line via a python ...
0
votes
0answers
28 views

Pytorch: Reduce forward prediction dimensions of GRU network / Improving Network Architecture

I am currently working on a GRU network to predict a time series, please note that I am completely new to machine learning and pytorch. Also I have never had a formal education in programming. This ...

1
2 3 4 5
9