Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
0
votes
1answer
17 views

How to reduce overfitting in a pre-trained network

I have a custom dataset with 10 classes and I am using a pre-trained resnet18 model from torch-vision. I can clearly see it's over-fitting because: the model is trained for 75 epochs with a batch size ...
0
votes
1answer
16 views

Multiclassification with large number of labels

I am attempting to build a classifier with a large input space of one hot encoded vectors. The output should be a vector of labels, with 10000 possible labels each. For example, the labels could ...
3
votes
1answer
17 views

Can I install Tensorflow and Keras on Cloud?

I will like to install Tensorflow and Keras on my PC. I use 32 bits OS. I learnt Tensorflow is not compatible with 32 bits. I cannot upgrade my OS to 64 bits since my hardware does not support it. I ...
2
votes
0answers
24 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
1
vote
2answers
28 views

How to make a neural network generalizes better?

I designed a neural network model with large number of output predicted by softmax function. However, I want categorize all the outputs into 5 outputs without modifying the architecture of other ...
0
votes
1answer
20 views

Where is the Backward function defined in PyTorch?

This might sound a little basic but while running the code below, I wanted to see the source code of the backward function: ...
0
votes
0answers
11 views

How do I efficiently load data from disk during training of deep learning models in pytorch?

I'm trying to train a deep learning model without loading the entire dataset into memory. My main question is, what's the best way of doing this? It seems like HDF5 is a common method that people ...
0
votes
0answers
13 views

Transfer learning by using vgg in pytorch

I am using vgg16 for image classification. I want to test my transfered model with the following code: ...
0
votes
0answers
17 views

What is the difference between register_buffer() and parameter.detach() in PyTorch?

I am writing a PositionalEmbedding() module which is an implementation based on "Attention Is All You Need" using PyTorch. According to the paper, there ...
1
vote
1answer
26 views
0
votes
0answers
14 views

Concatenating Encoder hidden states in LSTM pytorch

I am implementing a seq2seq autoencoder in pytorch: Q1) While it is true that we can keep the encoder as bidirectional, but can we keep the decoder as bidirectional as well(does it make any sense) if ...
0
votes
1answer
37 views

Why PyTorch is faster than sklearn models?

Recently, I get to know about the hummingbird library for Python. I trained a RandomForest on a 10M-sized dataset with 2 labels. With sklearn it was taking 450 ms for inference. But after converting ...
0
votes
1answer
27 views

Classifier using pytorch

I'm writing a demo code to predict a 2-class classification for a dataset of 10-D inputs. Below, function _data generates the data: ...
0
votes
0answers
15 views

Calculating key and value vector in the Transformer's decoder block

I am implementing the transformer model in Pytorch by following Jay Alammar's post and the implementation here. My question is regarding the input to the decoder layer. As shown in the diagram above, ...
1
vote
0answers
18 views

Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found here: https://medium.com/@ageitgey/natural-...
0
votes
0answers
20 views

Debugging Reinforcement Learning Model

I'm new to RL and I'm attempting to train an RL agent to play MsPacman in PyTorch. I've adapted the code from this tutorial on the PyTorch page for my problem. The DQN has the following architecture: <...
1
vote
0answers
41 views

how to implement squared hinge loss in pytorch

does anyone have any advice on how to implement this loss in order to use it with a convolutional neural network? Also, how should I encode the labels of my training data? We were using one hot ...
0
votes
1answer
37 views

How are the gradients of a Neural Network calculated just by matrix multiplication?

I would have expected some kind of derivative solving equation to be at work in order to back propagate the loss to each neuron. I hope my question is not too confused to answer. In the network below, ...
0
votes
0answers
8 views

Why does this model print Incompatible dilated conv1d layers?

I was trying to see the layers used in a Wavenet model for speech generation and I can't seem to make sense of the output layers printed by the TF model. Model is this: https://github.com/Rayhane-...
0
votes
0answers
6 views

Computing generalized variance for high-dimensional data

What are some practical methods to compute the generalized variance $|\Sigma|$ of a multivariate Gaussian, for high-dimensional data (d>200)? I'm having troubles computing the product of eigenvalues ...
0
votes
1answer
91 views

How do we implement a custom loss that backpropagates with PyTorch?

In a neural network code written in PyTorch, we have defined and used this custom loss, that should replicate the behavior of the Cross Entropy loss: ...
0
votes
1answer
13 views

Is there a loss function that measures the cross similarity between two 2D tensors?

Given two input tensors x1 and x2 with the shape [batch_size, hidden_size], let ...
0
votes
1answer
14 views

How to verify a CNN encoder works as expected?

I am using CNN as a part of kernel warping. The purpose here is to reduce input dimension (from N*M to K *1). The input data is not image data. I suspected that the CNN network might not work as I ...
1
vote
0answers
10 views

Why N-pair Loss (NIPS 2016) stops minimizing in Image retrieval task?

Currently, I am using a Deep Learning model to build a search engine for retrieving images. With a dataset of pairs of (image, description), I am using a ...
0
votes
0answers
11 views

How to decide whether to use categorical embeddings in a neural network?

I have a binary classification task with a whole slew of binary categorical features, one multiclass categorical and a few continuous features. I initially treated the categorical features using one-...
0
votes
0answers
7 views

Need for doing grad.zero_() after setting torch.no_grad() in pytorch

In the following lines of code, ...
0
votes
0answers
22 views

deep learning model gives same probabilities for the same class

I'm building a CNN neural network with Pytorch. Although the model accuracy is 85.6%, after getting image probabilities with torch.exp(output) and getting the top ...
0
votes
0answers
13 views

Text Classification One-shot learning (1sample/class ~1000 classes)

I'm working on making a classifier that given some description predicts a class. The two big issues that need to tackle is low number of examples (one example/class) and a large # of classes (~1000). ...
0
votes
1answer
26 views

Visualization Tools that can show proper branch and merge in Graphs

I want help finding the visualization tool can draw similar architecture as given in the image below Keras visualization produces something similar to this graph. But I'm working in Pytorch. I tried ...
0
votes
1answer
27 views

Cat Classifier becomes worse the more you train it

I am using a dataset from kaggle to train a feed forward neural-neteork with no convolutional layers. I wanted to try it this was as a learning exercise with Pytorch without Transfer Learning and ...
1
vote
2answers
21 views

Implementing training in PyTorch

I wish to accomplish the following task in PyTorch- I have the COCO dataset, wherein each data sample is used in training YOLO v3. After being processed by the model, the sample is to be deleted if ...
0
votes
0answers
14 views

Low GPU utilisation and High GPU memory

I am training a conv net for classifying 3 classes of images of size 512,512 using Pytorch framework. I have 3 Tesla V100s(16 Gb). My GPUs utilization is really low - <10% and GPU memory is really ...
0
votes
0answers
6 views

Transformers trainer sequence classification problem

I wanted to use XLMRobertaForSequenceClassification to classify a sequence into 1 or 0. <...
1
vote
0answers
116 views

Pytorch XLA to solve the spawn problems in a Colab Env

As reference only, here is my code It seems that torch.multiprocessing.set_start_method("spawn") can't be used in an Colab Env. Only 'fork' is allowed. I have ...
0
votes
0answers
12 views

Using pos_weight with BCEWithLogitsLoss to improve recall in a multi-label problem

I have a multi-label classification problem, and so I’ve been using the Pytorch's BCEWithLogitsLoss. I’d like to optimize my model for a higher F2 score, and so want to bias it to have greater recall (...
0
votes
0answers
12 views

Autoencoder to encode features/categories of data

My question is regarding the use of autoencoders (in PyTorch). I have a tabular dataset with a categorical feature that has 10 different categories. Names of these categories are quite different - ...
0
votes
1answer
20 views

Policy Gradient not “learning”

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
0
votes
1answer
118 views

Implementation of BERT using Tensorflow vs PyTorch

BERT is an NLP model developed by Google. The original BERT model is built by Tensorflow team there is also a version of BERT which is built using PyTorch. What is the main difference between these ...
1
vote
1answer
52 views

What's an appropriate datastore for variable length sequence data for PyTorch consumption?

I have a large number of sequences - potentially hundreds of thousands - each consisting of between 100 and 10,000 items, which each consist of about 5 floats. I need a datastore that can rapidly ...
0
votes
1answer
10 views

Unstable results in test mode with fractional max pooling in PyTorch

I make some variants of ResNet, originally found in TorchVision, modify them, train them and so on. What I have found is that even in .eval() mode, even if I load state right before evaluation, I ...
0
votes
0answers
13 views

Using sklearn scoring function in skorch neural net

I'm trying to use the skorch library to use PyTorch with sklearn. More specifically, I'm trying to use an sklearn scoring function (...
1
vote
0answers
8 views

How to convert subword PPL to word level PPL?

I'm using this formula to covert subword perpexity to word perplexity: PPL_word = exp(log(PPL_subword) * num_subwords / num_words) The question is do I need to ...
1
vote
0answers
10 views

Pytorch Distributed Computing - Recomendations/Resources/Courses?

I would like to get into some distributed computing for processing Pytorch CNN models. I am completely fresh in this field and want to get some recommendations as to where I should start researching ...
0
votes
1answer
18 views

Considering the output of a BLSTM in pytorch, what's the order of the elements?

I am currently using pytorch to implement a BLSTM-based neural network. I understand that the output of the BLSTM is two times the hidden size. However, I am currently unable to find out whether this ...
1
vote
0answers
17 views

Can be possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training bu I got this results: Loss goes up and ...
0
votes
0answers
82 views

Expected more than 1 value per channel when training, got input size torch.Size([1, xx])

Consider the following network snippet: ...
0
votes
0answers
9 views

Pytorch CNN with 2 image input and one output

I am new to CNNs and I am trying to design a CNN with 2 images as input and one value as the output. The images have shape (64*64*1) and contains a single line in them. All lines of the data set are ...
0
votes
0answers
17 views

The features from CNN and R-CNN for a region

I have a dataset of images with bounding boxes around the regions in the image. I do not need R-CNN to detect the regions as they are given in the dataset, but I need to extract the features of the ...
0
votes
1answer
34 views

Algorithm to calculate nerual network training time?

Before starting a new machine learning side project, it would be very useful to estimate how long it will take to run 1, 10, 100, 1k epochs. A crude estimate is more than sufficient (i.e. 1 epoch ...
1
vote
1answer
33 views

PyTorch is too heavy for deployment, is there a light weight pytorch for running neural networks?

We deploy PyTorch models in docker container, which massively increased the size of the docker container by more than 1G. But when we deploy the model the training has already been done, so ...

1
2 3 4 5
7