Questions tagged [deep-learning]
a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.
4,834
questions
5
votes
2
answers
274
views
Modeling uncertainty from known physics
I have an equation given by:
$$
\frac{\mathrm{d} s}{\mathrm{d} t}=4a−2s+\lambda(s)
$$
where, $a$ is an input constant and $\lambda$ is a non-linear term that depends on $s$.
I know that the true ...
- 51
0
votes
1
answer
56
views
Extract phrases/keywords that are SIMILAR to a python list of keyword/phrases, from a document
EDIT : If I had to match single worded phrases, I could first tokenize the text from the document and then calculate the cosine similarity of all the tokens with all the keywords from the ...
- 1,876
0
votes
0
answers
19
views
In recommendation systems, for methods that use backproporgation to get user feature, do they need to retrain the whole model when a new user is added
I'm tying to learn about recommendation systems recently. I have some deeplearning background so I focused more on machine learning based methods for recommendation systems. I see that a lot of paper ...
- 101
0
votes
0
answers
29
views
val_accuracy and val_loss not changing while training transformer
recently i have been trying to learn transformer and using it in caption-generator model.
While training for 4 hours val_loss and val_accuracy did not change. loss and accuracy for train_data was ...
- 135
0
votes
1
answer
20
views
Best practice labeling grouped anomalies for object detection
I would like to train object detection model (e.g. YOLO) for images that contain anomalies. The anomalies are essentially the holes in a surface of different sizes. How do I label correctly such ...
- 3
1
vote
2
answers
44
views
High accuracy on test and validation data but still can not predict on real data
Hello i am having a classification between two classes A and B and i have trained CNN model. I have high accuracy on all three set of data i.e training (98.7%) validation (99.3%) and test(98%) but ...
- 11
0
votes
0
answers
69
views
How do I interpret GRAD-CAM's feature attribution to time series zero-padding in a CNN classifier?
Problem setting: MTS Classification with CNN architecture
I have a multivariate time series (MTS) dataset that contains 30 features. The goal is to solve a classification problem on this MTS dataset. ...
0
votes
0
answers
92
views
Importance of sentinel token placement in T5
There is this paper that I have been trying to reproduce (https://arxiv.org/pdf/2205.11482.pdf) as part of my master's thesis. It uses T5 to learn facts from the training set where either the object ...
- 11
1
vote
0
answers
21
views
Influence functions on neural networks: Help with understanding of result and derivation
I'm working through a paper titled "Understanding Black-box Predictions via Influence Functions" where they introduce the notion of influence functions from robust statistics to approximate ...
- 11
0
votes
0
answers
45
views
How can I prevent mobilenetv3 from overfitting with less data?
So I have around 462 images and I can't really get more images. I am using a pretrained model of MobileNetV3 with the respective weights. I am facing a huge problem of overfitting and no real solution ...
- 47
0
votes
1
answer
34
views
How big is the threshold that is usually used in determining the convergence of loss values in deep learning?
In deep learning, one way to determine whether the training has converged is to observe the movement of the loss values over iterations or epochs. One can choose any $\epsilon$ threshold and any ...
- 13
1
vote
0
answers
46
views
MobileNet validation loss not decreasing over time
I am trying to train a MobileNetV2 on a custom dataset, to image Classification task.
Cardinality is 864 images, split in 70%/20%/10%, balanced between the 3 different classes.
Weights are pre-loaded ...
- 11
0
votes
0
answers
21
views
Embedding vector of MaskRCNN (Resnet with FPN)
I have a MaskRCNN model for instance segmentation with Resnet 50 - FPN backbone trained in detectron2. And I want to extract the embedding/feature vectors for visualizing input and hopefully detecting ...
0
votes
0
answers
12
views
Unable to reproduce result for CIFAR10 with ResNet50 backbone and SIMCLR self-supervised algorithm
I am recently working on self-supervised learning, particularly simclr paradigm. I found it hard to reproduce results on cifar10 with linear evaluation. I now get round 60%, while the paper reports 80%...
- 1
1
vote
2
answers
212
views
pillow cannot import / conda unexpected error
I created a conda environment previously and it worked fine with python and tensorflow. At that stage I used anaconda.
On a fresh install I am using miniconda since I now understand the conda commands ...
- 23
0
votes
1
answer
55
views
Transformers doubt
Basically here the $Q$,$K$ and $V$ are passed through a linear layer to obtain the actual $Q$,$K$ and $V$ for self attention mechanism and then we concatenate all of it.
My doubt is, I thought the $Q$,...
0
votes
1
answer
47
views
Understanding the ResNet paper
I am having trouble understanding the mathematical meanings behind the notations in the ResNet paper:
I believe that the function we are trying to optimize is the residual which is denoted as $\...
- 1
0
votes
2
answers
65
views
can training for too long lead to overfitting? I am not sure about the specifics of this
does training for a large number of epochs lead to overfitting? I am concerned about this as I am getting an accuracy of nearly 1 on val and training dataset when I am training for 50 epochs
- 15
3
votes
2
answers
152
views
How does variational autoencoders actually work in comparison to GAN?
I want to know about how variational autoencoders work. I am currently working in a company and we want to incorporate variational autoencoders for creating synthetic data. I have questions regarding ...
- 47
0
votes
1
answer
40
views
2 basic doubts on time series
Suppose say, I have to predict the cost of stock market. I have previous data and I have made it into the following Structure :
(Xt-3,Xt-2,Xt-1)--->(Xt=Yt)
Now the order of the above data points if ...
4
votes
1
answer
75
views
Working with time series data with several times stamps on a dates, and implementing machine learning
I'm trying to implement predictive analytics on a production data. my goal is to predict next downtime, it's reason and issues.
My data looks like below;
...
- 151
0
votes
0
answers
9
views
Help with trying to reimplement ADEC paper (clustering using AE and GAN). Encoder loss is not decreasing
I am trying to reimplement the ADEC paper (https://arxiv.org/abs/1909.11832) which mixes an autoencoder with a GAN network, but I am facing the issue that the encoder loss does not decrease.
I have ...
0
votes
0
answers
18
views
Given a 4d tensor for time series predicition
I have multiple time series datasets, which i want to train to an lstm model.
The shape of the training data is (735,2,5,4). 735 are the time steps, 2 are the two time Series datasets, 5 are the ...
1
vote
0
answers
26
views
what is the error here ? time series sliding windows
Given a list where each element is a dataframe, i want to create sliding windows in order to train a lstm model, but the problem is an error occurs.
Each dataframe is a time series with the 4 columns ...
- 13
0
votes
0
answers
14
views
Unsupervised SimCSE not Learning Good Representations
I'm using the unsupervised SimCSE approach to train a semantic embedding model on a corpus of tag documents. That is, ordered lists of tags concatenated into white-space seperated strings. I use a ...
- 101
1
vote
1
answer
63
views
What is the definition of convergence in the context of deep neural networks?
Suppose I have a feed forward neural network which approximates a value, say $Y_0$. The analytical value of $Y_0$ is given. The plot of the network approximation of $Y_0$ each step is given as follows....
- 13
0
votes
3
answers
83
views
How to properly do a k-fold cross validation?
I am trying to solve binary classification problem using deep neural networks. I want to compare different approaches (model architectures) and I have no hyperparameters which I want to tune. So my ...
- 13
0
votes
0
answers
15
views
Training a Face Recognition model using benchmark datasets
Is it recommend (or conventional) to train, for instance a face recognition method, using benchmark datasets such as XQLFW, RFW, CALFW, etc.? I would like to fine tune my model so it is more robust to ...
- 1
0
votes
1
answer
26
views
In the attention mechanism, why don't we normalize after multiplying values?
As this question says:
In scaled dot product attention, we scale our outputs by dividing the
dot product by the square root of the dimensionality of the matrix:
The reason why is stated that this ...
- 1,073
0
votes
0
answers
15
views
TypeError: TripletLoss.call() missing 1 required positional argument: 'neg'. Custom Training Loop
The below is the triplet loss. It's call method has 3 arguments
...
1
vote
1
answer
62
views
Why does my chess neural network not generalize well?
I'm trying to train a neural net to evaluate chess positions. For reference, the dataset I'm using can be found here. If you follow that link, you'll see that there are three CSV files - I'm only ...
- 79
0
votes
0
answers
15
views
How can a pullback dissimilarity on a nasty space be interpolated/approximated?
I have a map $\gamma : X \rightarrow Y$ that is expensive to compute. $X$ is a nasty, very non-Euclidean, not even manifold-like, space of variable-length and "structurally inhomogeneous" ...
- 101
0
votes
0
answers
19
views
Longer DNN training times when using evolutionary algorithms
I am comparing my deep neural network (DNN) performance when using 2 types of optimizers: gradient-based Adam (properly tuned) and a population-based optimization algorithm (e.g., genetic algorithm (...
0
votes
0
answers
30
views
CTC loss Expected input_lengths to have value at most 144, but got value 174
Can someone help me with CTC loss. I wrote a conformer model for ASR and to train encoder I need CTC loss. But when I train model I got error "Expected input_lengths to have value at most 144, ...
- 1
0
votes
1
answer
39
views
Bert model for document sentiment classification
I am trying to fine-tune a Bert model for sentiment analysis. Instead of one sentence, my inputs are documents (including several sentences) and I am not removing dots. I was wondering if is it okay ...
0
votes
0
answers
16
views
Using both gradient clipping and learning rate warmup, should gradients be clipped during the warmup, or only once the warmup phase has completed?
I'm training a network using both learning rate warmup and an adaptive gradient clipping method outlined here. Is there a general consensus or anything in the literature relating to whether gradients ...
- 1
0
votes
0
answers
102
views
Keras CNN model is throwing as error message as 'ValueError: Layer 'conv1d_12' expected 2 variables, but received 0 variables during loading'
Hope you're in good health and doing great.
I am trying to implement a CNN model to help predict kidney stones. Now, this model is running as expected on my local machine, but when I try deploying the ...
0
votes
0
answers
26
views
Minimizing the difference between two distributions with TensorFlow
Suppose that the encoding neural network of a variational autoencoder (VAE) outputs a distribution from which latent samples will be drawn. To do this, the layer ...
0
votes
0
answers
127
views
Transformer-based autoencoder generates same output and bad embeddings
I am trying to implement the transformer-based autoencoder presented in this paper: https://arxiv.org/abs/2210.08288
The paper seems rather vague to me and I do not fully understand how the model is ...
0
votes
1
answer
154
views
Can I use one-hot encoded output for segmentation in Pytorch, with focal and dice losses?
know that for classification using a neural network and CrossEntropy Loss, we need one-hot encoded output, but in PyTorch, the CrossEntropy loss does not accept one-hot encoded targets, and we should ...
- 101
0
votes
0
answers
33
views
The embedding output of bert
I want to get the embedding matrix of the Bert model (the input before the first block layers) to feed it into another architecture. I really appreciate it if you help me with that. Thanks
0
votes
0
answers
28
views
How do you find the coordinates of the bounding box for meituan Yolov6?
I'm trying to track a runner in a frame and eventually output velocity of the runner. The way that I am trying to do this is by finding the midpoint of the bounding box and then calculate the change ...
- 1
0
votes
0
answers
77
views
Getting a dimension mismatch when training dataset on openAI CLIP
Here's the code I've written:
...
0
votes
1
answer
40
views
Which model to use that can distinguish between names with the same words?
For my task, I need a model that can distinguish between job titles that contain the same words. BERT model "msmarco-MiniLM-L-12-v3" shows high cosine similarity for positions: "Data ...
- 101
0
votes
0
answers
191
views
How to copy and crop feature map in Unet?
I am confused about the principle of copy and crop in U-net, like the grey line shown above.
For example, the first grey line, how to convert a (64, 568, 568)(C,W,H) to a (128, 392, 392), did the ...
- 101
0
votes
0
answers
9
views
Downsides of predicting sentiment using a dictionary-based sentiment
I have a dataset of 30K tweets in English and I am using a sentiment-based dictionary/lexicon of 9K unique words that classifies each word into either positive or negative. However, some tweets have a ...
- 127
0
votes
1
answer
182
views
Dimensions of mel spectrogram
Can someone explain me dimensions in ASR? For example, if I have an audio, convert it to mel spectrogram and now I have a tensor of dimension [1, 128, 850]. Am I understand right that 128 - number of ...
0
votes
0
answers
9
views
How to handle datasets with multiple attributes in meta learning?
So far we have seen a meta-learning example with an image dataset (e.g., Omniglot) that has only one attribute. However, they have multiple attributes if we want to use non-image datasets (e.g., ...
- 101
0
votes
0
answers
9
views
How should I design my model architecture?
I have a dataset consisting of N presence points of a crop in a certain region. And I have 19 bioclimatic variables (temperature, precipitation, etc) which I extracted from .tif files. I have an input ...
- 1
0
votes
0
answers
20
views
Can sigmoid function applied on distance calculation of Self Organizing Map improve the accuracy?
I just thought how sigmoid function helps logistic regression making more accurate classification and wonder if it can be applied on distance calculation of Self Organizing Map and making more better/...
- 1