Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
2
votes
0answers
29 views

batched CrossEntropyLoss in pytorch

I'm wondering how to implement this with pytorch built-ins. I've got a 3 dimensional input of uints called policy. Most of the entries are zero, and if I were to L1 normalize this I would have a (...
1
vote
1answer
31 views

How does BERT and GPT-2 encoding deal with token such as <|startoftext|>, <s>

As I understand, GPT-2 and BERT are using Byte-Pair Encoding which is a subword encoding. Since lots of start/end token is used such as <|startoftext|> and , as I image the encoder should encode ...
1
vote
0answers
24 views

Where can I get pre-trained weights for character recognition?

I'm doing a simple pipeline and I need a digit recognition algorithm. There are definitely pre-trained weights out there for character recognition, but I can't find them. They can be in Keras, ...
1
vote
0answers
29 views

Compute loss gradient w.r.t. inputs of Bayesian Neural Network using Pyro

Suppose I inferred the parameters of all the posterior distributions for a BNN using Pyro. In particular, the implementation uses the HiddenLayer class: ...
3
votes
0answers
20 views

How to quantitatively evaluate raw neural network activations?

Below are the activations for 2 different predictions. These predictions are for different labels/classes. They are being run through a dense Keras NN (96.6% accurate) with 2 hidden layers and adam ...
1
vote
0answers
17 views

PyTorch Faster R-CNN ResNet50 - support for occlusion?

Currently I'm using the PyTorch model Faster R-CNN ResNet50. I instantiate this as follows: ...
2
votes
1answer
27 views

Multilingual Bert sentence vector captures language used more than meaning - working as interned?

Playing around with BERT, I downloaded the Huggingface Multilingual Bert and entered three sentences, saving their sentence vectors (the embedding of [CLS]), then ...
0
votes
1answer
27 views

How neural style transfer work in pytorch?

I am using this pytorch script to learn and understand neural style transfer. I understood most part of the code but having some hard time understanding some parts of the code. In ...
1
vote
1answer
24 views

Row-wise Jacobian with pytorch

Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$. Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$. I would like to ...
0
votes
2answers
34 views

PyTorch: How to use pytorch pretrained for single channel image

If I have to create a model in pytorch for images having only single channel. How can I transform my model to adopt to this new architecture without having the need to compromise the pre-trained ...
1
vote
0answers
27 views

Tiny Video Network

According to this: ...
0
votes
0answers
10 views

GIven some similar pictures,how to detect and mark the differences among these picture by deep learning

Now i have certain images ,they can be classfied into tow groups :right and false .However, two sorts just have small differnece in the detail . (for example ,I possess two maps. Their content is ...
1
vote
0answers
14 views

Churn rate prediction based on sequencial data

I am trying to build a machine learning model that can predict if a certain user will churn based on its historical static and dynamic data. The data looks like below: ...
0
votes
0answers
13 views

TypeError: Cannot handle this data type when changing the Dataset's data

trainloader = utilsxai.load_data_cifar10(batch_size=1,test=False) this is my dataset from torchvision ...
0
votes
1answer
38 views

How to calculate perplexity in PyTorch?

I am wondering the calculation of perplexity of a language model which is based on ...
1
vote
0answers
25 views

Place of backward and its relation with batches in PyTorch

I am implementing a dependency parsing model using PyTorch and little bit confused about the situation that I explained below. When calculating loss and backward the model; I tried different things. ...
0
votes
1answer
30 views

CrossMapLRN2d in pytorch

I had to convert a code written in pytorch to keras (with tensorflow backend). But there was this layer called CrossMapLRN2d which had no direct counterpart in Keras. So wanted to know what does this ...
0
votes
0answers
8 views

Jupyter notebook memory consumption while training vs script

Anyone seeing this issue where, when you're training a model with torch on Jupyter Notebook your GPU memory utilization is more than if it was a python script? Ran the same notebook through a script ...
0
votes
0answers
10 views

How to correctly mirror / inverse an encoder architecture for the decoder?

For the purpose of autoencoders, It seems to be common practice to build the decoder architecture in a way so that it mirrors the encoder architecture. However, there seem to be 2 approaches of doing ...
1
vote
0answers
15 views

Reinforcement learning converges for mean loss but not for each training data

Here I show a dummy example that represents my actual problem. My neural network (NN) receives one input and gives the probabilities for two output nodes. The code for the NN is: ...
0
votes
0answers
27 views

what should I do if my Neural network model stuck on high value loss?

I'm using neural nets in my projects. It's a regression problem where i have 3 features and I'm trying to predict one continuous value. I noticed that my neural net start learning good but after 10 ...
0
votes
0answers
10 views

Modifying network to handle images consistently

I'm modifying IRNet to work with the Cityscapes dataset. This network takes images as input and is supposed to output images that can be used as instance segmentation labels. IRNet originally uses ...
0
votes
0answers
18 views

my cifar10 kaggle result around 10% is there something wrong with my pytorch code/?

I'm quite new to pytorch so I want check is there something wrong I got final submission code score around 10% here is my code ...
0
votes
0answers
30 views

How do I disable libtorch warning

Recently I deployed a program using libtorch (PyTorch C++ API). The program run as expected but its gives me a warning. ...
1
vote
0answers
11 views
0
votes
1answer
16 views

How is loss computed for multiclass CNN with an output layer larger than the number of classes?

I have built a CNN in pytorch for classifying the Fashion-MNIST dataset (10 classes). The images are 28x28. I have constructed the final layer in my model as an output of 50. (i.e. $nn.Linear(...
0
votes
2answers
17 views

How to handle different categorical embedding sizes in hold out data set

I have a pytorch tabular dataset with zip code as a categorical embedding. I'm getting great results on the test set. When I go to run my hold out sample through, it errors out because I have more ...
0
votes
0answers
13 views

Slow inference speed on pytorch Object detection

Recently I was experimenting with different object detection techniques faster than mobilnet SSD v1 tensorflow api. I came across similar implementation in pytorch. But while execution I found that ...
-1
votes
1answer
55 views

What does the collate function in pytorch (geometric)?

I am creating a message passing neural network and have some issues with the dataset creation. In pytorch (geometric) it is recommended to create a dataset with the following class. I wonder what is ...
0
votes
1answer
85 views

Pytorch CrossEntropyLoss expected long but got float

have you done some research before asking the question? Yes. I have done a lot of online searching, and others had similar problems. There solution was to use .float() when entering into the loss ...
0
votes
1answer
82 views

Tutorial for restricted Boltzmann machine using PyTorch or Tensorflow?

I am trying to find a tutorial on training Restricted Boltzmann machines on some dataset (e.g. MNIST), using either PyTorch or Tensorflow. The few I found are outdated. Can you recommend any?
1
vote
0answers
23 views

Using a DCGAN to create an intrusion detection system

The TL;DR of my question is how do you write a discriminator and generator of a DCGAN in pytorch to accept a csv file instead of an image? I am attempting to partial recreate an experiment from the ...
1
vote
1answer
89 views

Fastest batch perspective transform for image augmentation

I need to do some augmentation for my training images for a neural networks. The problem is that even when loading batches in parallel, the augmentation is taking longer to perform than the network ...
0
votes
1answer
15 views

Why the first prediction of neural network in PyTorch is slower than following predictions?

So I have ResNet50 trained to classify images. For each prediction I track the time needed for it (input and model are moved to GPU): ...
0
votes
0answers
72 views

Adapting Pytorch tutorial “NMT from Scratch…” for dynamic RNN

I have taken the code from the tutorial and attempted to modify it to include bi-directionality and any arbitrary numbers of layers for GRU. Link to the tutorial which uses uni-directional, single ...
0
votes
0answers
24 views

Regression Task - Spark, PyTorch, TensorFlow or scikit

I know it's a broad question, sorry for that, but I'm still testing the waters with machine learning. I have a typical regression task (predict target numbers with the help of features x,y,z) and a ...
-1
votes
1answer
99 views

Softmax gives output vector whose sum is greater than 1 in Pytorch

I am a newbie to PyTorch. I was trying out the following network architecture to train a multi-class classifier. I used Softmax at the output layer and cross entropy as the loss function. However, the ...
0
votes
1answer
103 views

BERT - How Question answering is different than classification

Basically I am trying to understand how question answering works in case of BERT. Code for both classes QuestionAnswering and Classification is pasted below for reference. My understanding is: <...
4
votes
2answers
1k views

How to get sentence embedding using BERT?

How to get sentence embedding using BERT? ...
0
votes
0answers
16 views

How do I augment data after spliting traininng datset into train and validation set for CIFAR10 using PyTorch?

When classifying the CIFAR10 in PyTorch, there are normally 50,000 training samples and 10,000 testing samples. However, if I need to create a validation set, I can do it by splitting the training set ...
0
votes
0answers
9 views

DL classification loss lowers but doesn't go to zero

I'm trying to train a text classifier using pytorch and the model currently uses pretrained embeddings, a bi-lstm followed by a linear layer and dropout. When I start training the loss is at 60-70, ...
0
votes
1answer
28 views

What is a latent space vector?

I do not understand this about GANs. Apparently the Generator is supposed to receive a latent space vector as its input. Yet I couldn't find an example of how I can implement it in Pytorch. This is a ...
0
votes
0answers
13 views

How can I use MIT CBCL dataset in python

I have downloaded the MIT CBCL face dataset, but I can't find any code online on how to use it in python. the MIT data set is in two formats .greynorm and .pgm format which I don't know how to use any ...
0
votes
0answers
18 views

CNN model with transfer learning not performing, training loss is still high, test accuracy is very low

Hi I'm trying to train a cnn model with transfer learning, and I am not able to get a good test accuracy (14%) - I don't know why it doesn't work for me. ...
1
vote
1answer
49 views

Supported GPU for Pytorch

Question Which GPUs are supported in Pytorch and where is the information located? Background Almost all articles of Pytorch + GPU are about NVIDIA. Is NVIDIA is the only GPU that can be used by ...
0
votes
1answer
13 views

Why is the variance going down so much in this weight initialization problem(using pytorch)?

first look at this example >>> x = t.randn(512) >>> w = t.randn(512, 500000) >>> (x @ w).var() tensor(513.9548) it makes sense that ...
0
votes
1answer
63 views

Dealing with input images of different shapes in PyTorch

I've started to work with a leaf classification dataset on Kaggle. All input images have different rectangular shapes. I want to transform the input into squares of a fixed size (say, 224x224) with a ...
1
vote
0answers
69 views

Pytorch lstm model very high loss in eval mode against train mode

I am using a Siamese network with a 2-layer lstm encoder and dropout=0.5 to classify string similarity. For each batch, I am randomly generating similar and ...
0
votes
1answer
27 views

PyTorch CNN network outputs homogeneous results

I am a beginner at data science and I got a project where I want to do nlp via a convolutional neural network in PyTorch. The problem is that regardless of what comes out of the convolutional layers, ...
0
votes
0answers
9 views

play session from ratings dataset in Movie20 M

I need to extract listening sessions from the ratings dataset which has the columns cols = [userId movieId rating timestamp] timestamp is just a number for eg 1112486027 listening sessions are ...