All Questions
Tagged with pytorch tensorflow
73 questions
0
votes
0
answers
19
views
tensorflow to pytorch weight transfer
There is a modified efficientnet TF model, that I'm trying to simulate in pytorch. I have made the architecture changes to the model in pytorch, dumped the TF model weights, and loaded them back in ...
- 101
0
votes
0
answers
25
views
How to Prepare Data for U-Net Model Training with .tif Images
I'm new to image segmentation and trying to train a U-Net model. I have a dataset consisting of .tif satellite images and their corresponding annotations. Here is a sample of my data:
...
- 1
0
votes
1
answer
27
views
Where should I learn pytorch from?
I'm a undergraduate student. I've coded a three-node neural network (that works) based on my professor's guidance. However, I'd like to pursue a career in AI and Data Science, and I'd like to teach ...
0
votes
0
answers
7
views
Vocal Tract Length Perturbation (VTLP) code
I'm looking for an example of code that uses the VLP (Vocal Tract
Length Perturbation) technique to improve speech recognition.
Regards
- 1
1
vote
1
answer
59
views
wierd neural network approache
I'm working on a problem where I need to create a neural network to optimize the seating arrangement for 24 unique individuals in a 6x4 grid, minimizing conflicts between adjacent (up,down,left,right) ...
- 101
0
votes
1
answer
30
views
How to get the closest samples to time series dataset?
I have a deep learning time series classification model.
I want to understand if the model failed to classify, due to missing or incorrupt training inputs.
For simplicity let's say we have a training ...
- 527
1
vote
1
answer
148
views
How to add a new item in the embeddings vocabulary?
Imagine you have trained a model containing an Embedding layer.
Your model performs well and you're happy with your embedding.
Then, suddenly, you want to add a new item in your vocabulary. In other ...
- 111
0
votes
0
answers
15
views
Is it fair to say that Hausdorff Distance (HD) focuses on low level details while dice score (DSC) high level
I wonder if its make sense to say that Hausdorff Distance (HD) measures low-level details while dice score (DSC) focuses on high levels.
If you could cite a paper, I would appreciate it.
- 127
0
votes
0
answers
332
views
How to use pretrained encoder for customized Unet
if you have a standard Unet encoder such as resnet50, then it's easy to add pertaining to it. for example:
...
- 127
0
votes
1
answer
480
views
Effect of hyperparameters: the hidden size, layers, MLP size number of heads on Transformer
Is there any paper that explains the effect of hyperparameters:
hidden size
Number of layers
MLP size
number of heads
on Transformer performance. I found some explanation on the web but I need ...
- 127
0
votes
1
answer
85
views
Query about Sigmoid activation function calculation
While applying sigmoid activation function (in finding y label), I have calculated it as below:
y = 0.35 + (0.8 * 0.1) + (0.3 * 0.6) + (-0.2 * 0.4) = 0.53
sigmoid_y = 0.625
how do we take threshold ...
- 67
4
votes
1
answer
767
views
Should a Learning Rate Scheduler adjust the learning rate by optimization step (batch) or by epoch?
In PyTorch doc, it suggests
torch.optim.lr_scheduler provides several methods to adjust the learning rate based on the number of epochs.
However, from other ...
- 392
1
vote
1
answer
666
views
How to handle OOV in non-contextual embedding (word2vec, Glove, FastText)?
how non-contextual embedding (Word2Vec, Glove, FastText) handle OOV (incase if given word is not available in vocabulary)
- 67
2
votes
2
answers
433
views
What is the ELMO approach to learn contextual embedding?
BERT, GPT, and ELMo used the contextual embedding. but, their approach of learning contextual embedding is different.
so, what is the ELMo approach to learn contextual embedding?
- 67
0
votes
1
answer
1k
views
Difference between Word2Vec and contextual embedding
am trying to understand the difference between word embedding and contextual embedding.
below is my understanding, please add if you find any corrections.
word embedding algorithm has global ...
- 67
1
vote
2
answers
1k
views
Will ML frameworks work with integrated graphics card in laptop?
I'm CS student. I'm just getting started in data science and machine learning. Can i use laptops with integrated graphics card in the data science and machine learning projects?
Will pytorch and ...
- 11
3
votes
2
answers
1k
views
Training tricks for increasing stability in mixed precision
I would love to be able to use automatic mixed precision more extensively in my training, but I find that it is too unstable and often ends in NaNs. Are there any general tricks in training that ...
- 58
1
vote
0
answers
389
views
Dataset Format for fine tuning deepset/roberta-base-squad2 hugging face transformer model
I have been trying to fine tune the roberta model for QnA to my specific domain (healthcare).
I am unable to find the correct way to provide the dataset format to the tokenizer in order to fine tune ...
- 111
0
votes
1
answer
22
views
What says the output of autoencoder?
What is the meaning of output of autuencoders? Can we say it is the noise removed version of actual dataset and should it be symmetrical?
- 21
0
votes
1
answer
153
views
Using 'Mlxtend' with 'TensorFlow' or 'Pytorch'
Is it possible to create a simple stacking implementation for regression with 'Mlxtend' using models created by 'TensorFlow' or 'Pytorch' however the documentation only supports examples that contain '...
- 101
0
votes
0
answers
2k
views
Pytorch: Which class weights I need to pass to BCELoss
I'm trying to build a Resnet model with Sigmoid with BCELoss lose. Since my data is imbalance, I guess I need to use "class weights" as an argument for the "BCELoss". But which ...
- 127
1
vote
0
answers
23
views
If an FCN accept rectangular image as input or has to be square?
Some say that for FCN it doesn't matter if the input image is rectangular the only thing matters that the size must be constant ...
- 11
1
vote
0
answers
105
views
Failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered [closed]
While running a code I am encountering this error any ideas on how to resolve this
- 11
0
votes
0
answers
73
views
How can I reshape (16,) and (3,112,112) shape to the single (16,3,112,112)? See code below
This is my code;
for img_loc in list(self.train_data)[idx]:
images_set.append(self.load_ucf_image(img_loc))
print(images_set)
And, this is its output
...
0
votes
1
answer
853
views
Cuda for PyTorch and Cuda for Tensorflow
I want to install PyTorch and for that I visited PyTorch official website, and they give me a command to install it with Cuda:
...
- 105
1
vote
2
answers
46
views
CNN: visualize a model using its description
i created a Resnet model, which i want to show in a presentation, but i don't know how to visualize what i have done?
Is there a tool or something to get a graphic from the description of my model.
...
- 113
0
votes
1
answer
4k
views
CNN: training accuracy vs. validation accuracy
I just finished training two models, while the one is pretrained and the other trained from scratch and created two diagrams afterward with their data, but as I am very new to machine learning, I don'...
- 113
2
votes
1
answer
711
views
0
votes
1
answer
16
views
The learning_rate in TensorFlow code is the sum learning rate of a batch or the learning rate of a data?
import tensorflow as tf
batch_size = 10
learning_rate = 0.001
tf.train.AdamOptimizer(learning_rate)
So the question is:
The ...
- 711
0
votes
1
answer
946
views
Memory issue when trying to initiate zero tensor with pytorch
I am facing a memory issue while trying to initialize a torch.zeros -
torch.zeros((2000,2000,3200), device=device)
Getting the following error:
...
1
vote
0
answers
147
views
Which One is the Best Way to Create Training Sequences for LSTM-based Class Prediction on Time-series Data?
Let's say I have time-series data in the following way. I need to create training sequences of a fixed length as an input to my LSTM model on PyTorch.
...
- 171
0
votes
0
answers
63
views
Trying to extend this code to include additional feature volume (in addition to adj close) RNN to predict adj close
I read this article on medium
https://medium.com/swlh/a-technical-guide-on-rnn-lstm-gru-for-stock-price-prediction-bce2f7f30346
prep
...
- 131
4
votes
1
answer
3k
views
How does the Transformer predict n steps into the future?
I have barely been able to find an implementation of the Transformer (that is not bloated nor confusing), and the one that I've used as reference was the PyTorch implementation. However, the Pytorch ...
- 43
1
vote
1
answer
400
views
Normalized 2D tensor values are not in range 0-1
Below function takes in 2D tensor and normalizes it using broadcasting .The issue is except all values to be in range 0-1 but the result has values outside this range . How to get all values in 2D ...
- 1,511
0
votes
1
answer
34
views
How to deal with big output values after classification layer during training?
In AI libraries such as, Tensorflow, Keras etc., how the big output numbers are dealt during training process? For example, for classification layer, it will have some outputs such as ...
- 109
0
votes
1
answer
458
views
Mask R-CNN Background Subtraction Implementation
I am currently attempting to reimplement a paper on fall detection (https://ieeexplore.ieee.org/abstract/document/9186597). It requires a background subtraction algorithm called Mask R-CNN. Are there ...
- 1
1
vote
0
answers
14
views
Common human-writable text format for specifying the architecture (untrained) of a neural network?
I am curious if there is a common human-writable text format for specifying the architecture (untrained) of a neural network.
There are informal notations such as described here.
I am aware of ONNX ...
- 111
2
votes
1
answer
2k
views
How to train a neural network on multiple objectives?
I have a multi-class neural network classifier that has K classes(products). For every row, only one of the classes will be 1 at a time. Now, this approach works fine if I have only 1 objective to ...
- 133
0
votes
1
answer
420
views
How to pass input to deep learning models for Multiple choice question answering problem?
I'm currently working on a multiple-choice question answering system. The training set consists of a question, answer and 4 options and I need to predict the correct answer among 4 options. Sometimes ...
- 425
1
vote
0
answers
168
views
What does it mean to say convolution implementation is based on GEMM (matrix multiply) or it is based on 1x1 kernels?
I have been trying to understand (but miserably failing) how convolutions on images (with height, width, channels) are implemented in software.
I've heard people say their convolution implementation ...
- 123
3
votes
1
answer
1k
views
How to choose between Tensorflow and Pytorch?
Recently I've been working on a pretty vanilla ANN model in Python with sklearn (and its preprocessing pipeline), mostly in jupyterhub notebooks if that matters.
I am considering changing the ...
- 2,444
1
vote
1
answer
41
views
Right Package for Federated Learning
Could Someone list the pros and cons with respect to using federated learning with the following packages:
TensorFlow federated
PySyft
Are there certain tasks which are specific to either or is one ...
- 482
0
votes
0
answers
43
views
How to insert sentences into training data which has 2 words, 3 words 4 and 5 etc into training data?
I have a set of sentences which each contain 2 words, 3 words, 4 words, 5 words etc. When I am trying to give the training data only the first two words in a sentence it is not accepting it. It is ...
- 181
2
votes
0
answers
231
views
Do learning rate scheduler have any significant improvement or redundant on Adam optimizer?
As in paper, Adam optimizer is adaptive learning rates algorithm.
Is learning rate scheduler become redundant when use with Adam and AdamW ?
Is it best practices to use learning rate scheduler with ...
- 243
1
vote
1
answer
316
views
Multiclassification with large number of labels
I am attempting to build a classifier with a large input space of one hot encoded vectors. The output should be a vector of labels, with 10000 possible labels each. For example, the labels could ...
- 11
2
votes
1
answer
72
views
Can I install Tensorflow and Keras on Cloud?
I will like to install Tensorflow and Keras on my PC. I use 32 bits OS. I learnt Tensorflow is not compatible with 32 bits. I cannot upgrade my OS to 64 bits since my hardware does not support it. I ...
- 171
3
votes
0
answers
129
views
AlexNet Research Paper VS PytTorch and Tensorflow implementation
I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain.
In the research paper, ...
- 233
1
vote
2
answers
1k
views
Is there a loss function that measures the cross similarity between two 2D tensors?
Given two input tensors x1 and x2 with the shape [batch_size, hidden_size], let ...
- 131
1
vote
1
answer
4k
views
Implementation of BERT using Tensorflow vs PyTorch
BERT is an NLP model developed by Google. The original BERT model is built by the TensorFlow team, there is also a version of BERT which is built using PyTorch. What is the main difference between ...
- 153
2
votes
0
answers
342
views
PyTorch equivalent of tf.Data
I've created a pipeline using tf.Data (or more accurately a mix of Pandas and then tf.Data). ...
- 440