Questions tagged [pytorch]

Pytorch is an open source library for Tensors and Dynamic neural networks in Python with strong GPU acceleration. For details, see https://pytorch.org.

Filter by
Sorted by
Tagged with
0 votes
2 answers
5k views

Pytorch CrossEntropyLoss expected long but got float

have you done some research before asking the question? Yes. I have done a lot of online searching, and others had similar problems. There solution was to use .float() when entering into the loss ...
0 votes
0 answers
15 views

Semantic segmentation sometimes give bad result

I'm training Unet+MobileNetV3 for semantic segmentation objects on real photos using custom dataset and get strange results. I have already accumulated pretty big dataset and constantly improve it by ...
0 votes
1 answer
78 views

Is it possible to train a single input->neuron->relu->neuron->relu for input > 0.5?

The neural network is simply: y=max(max(x*w+b,0)*v+d,0) w,b is weight and bias of first neuron. v,d is weight and bias of second neuron. If data is for example: <...
2 votes
1 answer
174 views

Row-wise Jacobian with pytorch

Suppose I have $f:\mathbb{R}^{d_i}\to\mathbb{R}^{d_o}$. Let $X \in \mathbb{R}^{n \times d_i}$ and I apply $f$ to each row of $X$, obtaining $Y = f(X) \in \mathbb{R}^{n \times d_o}$. I would like to ...
2 votes
1 answer
116 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
0 votes
0 answers
5 views

Seeking ML Model Recommendations for Enhancing OCR on Corrupted Text Images

I am working on a project where I need to perform Optical Character Recognition (OCR) on text-based images. However, these images are corrupted in various ways (e.g., blurred, distorted, low ...
0 votes
0 answers
8 views

Safetensors are slower than torch.load. Why?

Lately, I have been testing safetensors. It claims that the safetensors are faster and safer than pickle. So I did my own comparison with the code below: ...
2 votes
1 answer
2k views

Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same

I am trying to run this notebook on Apple M1 (1st gen) running MacOS 12.4, libs freeze: ...
0 votes
0 answers
24 views

Is creating a new model through the weighted average of the best models in hyperparameter-tuning a (good) thing?

I'm currently working on a forecasting project with python/pytorch, I coded a very simple hyperparameter-tuner that randomly picks hyperparameters and saves the model and performance stats. I'm ...
1 vote
0 answers
22 views

How to apply this schematic of a CNN?

I'm trying to apply a model from a paper to my problem, however I get very poor results (R² = 0.1 for regression). I think I don't understand the schematic drawing of the CNN used in the paper. Could ...
3 votes
1 answer
271 views

Policy Gradient not "learning"

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
0 votes
1 answer
372 views

Linear Regression in Pytorch-vanishing gradient with Softmax

I am implementing a non-linear regression using neural networks with one single layer in Pytorch. However, using an activation function as ReLu or Softmax, the loss gets stuck, the value does not ...
1 vote
1 answer
15 views

Extraction of name from phonetic transcription

I have a use case where I want to extract the name from the phonetic transcription. For example if the phonetic transcription is - “m a j n e j m ɪ z s ʌ m i ɹ z o w ʃ i”, the output should be the ...
0 votes
0 answers
36 views

reverse summary matching

How can we develop a model to assess the relationship between a summary and a text, enabling it to determine their coherence but in reverse? Also, how to make an inference We have list of texts (2000) ...
-1 votes
0 answers
20 views

Environment setup advice [closed]

For frameworks, PyTorch and TensorFlow seem like the go-to choices. Which one would you recommend for ST-GNNs, and are there any specific libraries (like PyTorch Geometric) that are particularly ...
-1 votes
1 answer
51 views

Why is my NN model's prediciton for y= sinc(x) function showing symmetric?

...
1 vote
1 answer
95 views

Neural Net not able to learn simple analytical equation

I am currently making my first attempts with Pytorch. I am trying to solve a simple equation with a neural net. Analytically solved, the result of my neural net shall look like this: $$ y = \frac{x_5}{...
0 votes
1 answer
20 views

How to get the closest samples to time series dataset?

I have a deep learning time series classification model. I want to understand if the model failed to classify, due to missing or incorrupt training inputs. For simplicity let's say we have a training ...
1 vote
1 answer
27 views

How to add a new item in the embeddings vocabulary?

Imagine you have trained a model containing an Embedding layer. Your model performs well and you're happy with your embedding. Then, suddenly, you want to add a new item in your vocabulary. In other ...
0 votes
0 answers
16 views

Custom loss function for collinearity of 3 embeddings

I am trying to implement a loss function that takes as input 3 embeddings and output a value that is proportional to the collinearity of the embeddings. This is to shape the latent space of a ...
0 votes
1 answer
323 views

GPU shows 0 utilization even when tensors and model are mounted on the gpu?

I am trying to run some PyTorch scripts on a remote GPU server. While calling the script in the ubuntu terminal i start as:...
0 votes
1 answer
311 views

Loading a Model with weights and optimizers without creating an instance in PyTorch

I recently downloaded Camembert Model to fine-tune it for my purpose. Upon unzipping the file the contents are: Upon loading the model.pt file using pytorch: ...
2 votes
1 answer
190 views

Why yolo4 pytorch re-training loss seems high as like first time training?

I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git. I generated checkpoints by giving ...
1 vote
1 answer
56 views

"model.to('cuda:6')" becomes (nvidia-smi) GPU 4, same with any other "cuda:MY_GPU", only "cuda:0" becomes GPU 0. How do I get rid of this mapping?

Strange mapping: example In the following example, the first column is chosen in the code, second column is the one that does the work instead: 0:0 1234 MiB 1:2 1234 MiB 2:7 1234 MiB 3:5 2341 MiB 4:1 ...
0 votes
1 answer
109 views

Transformers Trainer: "RuntimeError: module must have its parameters ... on device cuda:6 (device_ids[0]) but found one of them on device: cuda:0"

I ask this since I could not fix it with the help of: Stack Overflow RuntimeError: module must have its parameters and buffers on device cuda:1 (device_ids[0]) but found one of them on device: cuda:2 ...
1 vote
1 answer
255 views

Should weight distribution change more when fine-tuning transformers-based classifier?

I'm using pre-trained DistilBERT model from Huggingface with custom classification head, which is almost the same as in the reference implementation: ...
0 votes
0 answers
14 views

How to detect abnormal fetal head size with image classification?

I'm writing Python code to predict fetal head circumference 10mm range using classification. The model will train to classify a fetal head image into a range (e.g., 50–60 mm) representing its ...
0 votes
1 answer
23 views

Performance difference between two equivalent ML codes

Using the two Python libraries GPyTorch and scikit-learn to perform Gaussian Process Regression (GPR) for a machine learning task, I have encountered a problem I failed to solve during the last days. ...
1 vote
2 answers
29 views

Image segmentations vs image detection

If I need to detect on an image some objects and we are only interested in counting them, between image segmentation and object detection which one would you think would yield best results in terms of ...
0 votes
0 answers
9 views

Help with a MaNet finetuning (binary semantic segmentation task)

Introduction: I am currently working on a computer vision problem, I have satellite images and I have to detect a particular archeological structure (Tell). I have access to the previously made ...
3 votes
2 answers
15k views

Supported GPU for Pytorch

Question Which GPUs are supported in Pytorch and where is the information located? Background Almost all articles of Pytorch + GPU are about NVIDIA. Is NVIDIA the only GPU that can be used by Pytorch? ...
0 votes
1 answer
433 views

Mask R-CNN Background Subtraction Implementation

I am currently attempting to reimplement a paper on fall detection (https://ieeexplore.ieee.org/abstract/document/9186597). It requires a background subtraction algorithm called Mask R-CNN. Are there ...
2 votes
2 answers
638 views

Loss function when the output is a single probability

I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1] ...
0 votes
0 answers
6 views

Is it fair to say that Hausdorff Distance (HD) focuses on low level details while dice score (DSC) high level

I wonder if its make sense to say that Hausdorff Distance (HD) measures low-level details while dice score (DSC) focuses on high levels. If you could cite a paper, I would appreciate it.
1 vote
2 answers
82 views

Classification with a lot of the classes

I’m trying to make model which will classify text into about 500 different classes. I think that I have to customize architecture of the Pooling Classifier which looks now like this: ...
3 votes
1 answer
137 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
1 vote
1 answer
5k views

Converting a Keras model to PyTorch

I have a Keras h5 file that I want to load into the same model but this one is created using PyTorch. Is ONXX a viable intermediary option? What else can I use?
0 votes
2 answers
38 views

Why is the variance going down so much in this weight initialization problem(using pytorch)?

first look at this example >>> x = t.randn(512) >>> w = t.randn(512, 500000) >>> (x @ w).var() tensor(513.9548) it makes sense that ...
0 votes
2 answers
514 views

Pretrain RoBERTa model with new data using PyTorch library

I've pretrained the RoBERTa model with new data using a 'simpletransformers' library: ...
0 votes
0 answers
25 views

CNN segmentation models: class weights specification on IoU metric

I am building a MANet model using pytorch lightning. For getting the model I use the library segmentation models. As my objective is to do binary semantic segmentation, during the test phase I ...
3 votes
1 answer
1k views

How to choose between Tensorflow and Pytorch?

Recently I've been working on a pretty vanilla ANN model in Python with sklearn (and its preprocessing pipeline), mostly in jupyterhub notebooks if that matters. I am considering changing the ...
0 votes
0 answers
22 views

why does my multi-modal model can not learn anything?

I have a multi-modal model. I want to train it using the Pytorch Framework. I have a balanced dataset. I have approximately 150 samples for each client. (I had preprocessed my text data.) when I train ...
2 votes
1 answer
495 views

A model that only works by setting all initial weights to zero

In this model from MusicNet, they set the initial weights of their neural network to all zeros. ...
0 votes
0 answers
9 views

Relationship among different classifiers of a model in multiclass problems

Suppose we are fitting a LogisticRegression model with scikit-learn, or the same model with pytorch. In multiclass problems, the strategy OneVsRest will fit a different classifier for each of the ...
0 votes
1 answer
265 views

How to decide the padding size and stride size in CNN

In CNN in 2d, what situation is the size of the padding and stride changed in? So far, I could make sense of the basic concepts with padding and stride. Padding and stride can be used to adjust the ...
0 votes
1 answer
88 views

How to implement a customized DataLoader that inherits pytorch's one?

I need to implement a customized DataLoader, that inherits from torch.data.utils.DataLoader. I have searched it for half hour, but there is no example or doc about this. What methods of it should I ...
0 votes
1 answer
105 views

Unstable results in test mode with fractional max pooling in PyTorch

I make some variants of ResNet, originally found in TorchVision, modify them, train them and so on. What I have found is that even in .eval() mode, even if I load state right before evaluation, I ...
0 votes
1 answer
391 views

PyTorch mat1 and mat2 shapes cannot be multiplied (100x200 and 100x9922)

I am trying to make a BiLSTM language model and am having some issues. Model ...
2 votes
1 answer
2k views

Is GEMM used in Tensorflow, Theano, Pytorch

I know that Caffe uses GEneral Matrix to Matrix Multiplication (GEMM) which is part of Basic Linear Algebra Subprograms (BLAS) library for performing convolution operations. Where a convolution is ...
0 votes
1 answer
96 views

Appropriate input size for nn.Embedding

I’m quite new to using Pytorch and deep learning. What size of unique categories of a categorical variable is appropriate for applying the nn.Embedding ideally (best practices)? for example, if a ...

1
2 3 4 5
15