Questions tagged [deep-learning]
a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.
4,843
questions
1
vote
1
answer
202
views
Early vs Late Fusion in Multimodal Convolutional Neural Networks
As for Early, Middle, and Late Fusion in Multimodal Convolutional Neural Networks
What is difference among them? Are there nice document or arcile which are describing these.
Best regards.
CommunityBot
- 1
2
votes
2
answers
961
views
How to remove background (watermark) logo from image
I have been scratching my head for a while. What I have is a scanned PDF document with text and water marked logo at the back as in the below image.
I want to do OCR over this, which becomes very ...
CommunityBot
- 1
0
votes
1
answer
86
views
Why a CNN with decreasing filter layers sizes could perform better than a "regular one" with increasing sizes?
I did dozens (or probably hundreds) of tests and the best result with less total parameters(4 times or less) was a decreasing filter layers size architecture. This is a CNN for multiclass image ...
CommunityBot
- 1
3
votes
2
answers
1k
views
Neural network approach to the cocktail party effect
Imagine you have 2 people at 2 different microphones but in the same room. Each microphone is going to pick up some sound from the other person. Is there a good neural network based approach to ...
CommunityBot
- 1
0
votes
1
answer
343
views
predict_classes() returning only 0 or 1 for multiclass image classification
I am trying to build a multi class image classifier but the only returns 0 or 1 .
Why is it not returning "Rock" , "Paper" , "Scissor" ?
and why only 0 and 1 but not 2?
CODE:
...
CommunityBot
- 1
0
votes
1
answer
82
views
What can we learn from visualizing Feature Maps
I have the following classification model (dogs vs cats):
...
CommunityBot
- 1
2
votes
2
answers
407
views
Estimating the uncertainty of regression models
Given a regression model, with n features, how can I measure the uncertainty or confidence of the model for each prediction?
Suppose for a specific prediction the accuracy is amazing, but for another ...
CommunityBot
- 1
0
votes
1
answer
148
views
Unbalanced dataset on image classification, is it better to lose samples and balance it?
I am dealing with a binary image classifier. I'm using a CNN to predict if an image is positive or negative. The problem is that the positive class represents only the 2% of the total samples. In this ...
CommunityBot
- 1
0
votes
1
answer
60
views
how to define a linear function WX+b in pytorch?
I am practicing pytorch. I want to define a linear function Y=WX+B for inputs shape as (3,32,32) and output, the same shape i.e. (3, 32, 32). I defined m network as:
...
- 1,015
2
votes
1
answer
330
views
Can I analyze Video in AI?
For those who do AI or Machine Learning, is it possible to analyze video? For example, here’s a tidbit from an article I’ve been reading:
in basketball, data can help coaches determine if a player ...
3
votes
1
answer
2k
views
Is finetuning from a pretrained model always better than training from scratch?
At the worst case scenario, we could treat the pretrained weights as a random initialization, same as what we would do for training from scratch, right? If that is the case, then wouldn't it be better ...
CommunityBot
- 1
3
votes
3
answers
134
views
In Neural Nets, why Use Gradient Methods as Opposed to Other Metaheuristics?
In training deep and shallow neural networks, why are gradient methods (e.g. gradient descent, Nesterov, Newton-Raphson) commonly used, as opposed to other metaheuristics?
By metaheuristics I mean ...
- 123
3
votes
1
answer
308
views
Why does joint embedding of word and images work?
I often see some papers where the authors do point-wise multiplication of word and image embedding (e.g the image below).
Why does this implementation works? I do not understand.
CommunityBot
- 1
0
votes
0
answers
12
views
How is it called when instead of creating predective models finding patterns in observed data (ML) you tried to guess the model theorically...?
I'm a college student appasionated of machine learning and I've decided to my bachelor thesis about it. I thought that as an interesting introduction to machine learning, I could introduce it by ...
1
vote
2
answers
342
views
How gradients are flown back to Network in siamese architecture? How weights of all CNN models are same even when using different models
TL;DR: Intuition behind the gradient flow in Siamese Network? How can 3 models share the same weights? And if 1 model is used, how Gradients are updated from 3 different paths?
I am trying to build a ...
CommunityBot
- 1
1
vote
3
answers
315
views
Understanding the concept vanishing gradient and exploding gradient problem in terms of training data
I'm trying to figure out the essence of the concepts "vanishing gradient and exploding gradient problem" in terms of real-world input-output training examples instead of in terms of the properties of ...
CommunityBot
- 1
1
vote
1
answer
218
views
LSTM with input of actual time step
I'm working on an implementation of LSTM neural network to forecast energy consumption. I have a dataset with load, series of weather parameters and indicator of it's bank holiday or not.
I first ...
CommunityBot
- 1
1
vote
1
answer
518
views
Should output data scaling correspond to the activation function's output?
I am building an LSTM with keras which have an activation parameter in the layer. I have read that scaling on the output data ...
CommunityBot
- 1
2
votes
2
answers
556
views
Neural network model for sparse multi-class classifier on Tensorflow
The problem I'm trying to solve is the following:
the data is Movielens with N_users=6041 and N_movies=3953, ~1 million ratings.
For each user, a vector of size N_movies is defined, and the values ...
CommunityBot
- 1
0
votes
2
answers
280
views
Self-Attention Summation and Loss of Information
In self-attention, the attention for a word is calculated as:
$$
A(q, K, V) = \sum_{i} \frac{exp(q.k^{<i>})}{\sum_{j} exp(q.k^{<j>})}v^{<i>}
$$
My question is why we sum over the ...
CommunityBot
- 1
1
vote
1
answer
517
views
How to compute f1_score for multiclass multilabel classification
I have used one hot encoder [1,0,0][0,1,0][0,0,1] for my functional classification model.
The predicted probabilities for test data ...
CommunityBot
- 1
0
votes
1
answer
187
views
How many bounding boxes does the YOLOv6 model predict in total before thresholding?
I understand that the YOLOv5 model predicts 25200 bounding boxes between all 3 levels of output. How many does the YOLOv6 model predict, if the input resolution is 640x640?
CommunityBot
- 1
0
votes
1
answer
416
views
Validation loss not decreasing using dense layers altough training and validation data have the same distribution
I have a problem that I have great difficulties understanding the concept that leads to these results. I use a keras dense layer to map 13 input features to 3 output labels. During the training, the ...
CommunityBot
- 1
0
votes
0
answers
16
views
Cost function looks like the real math which is responsible for actually working on out problem statement If I talk on a whole and on a surface level?
Looking at the cost function for say linear regression, apart from changing the weight or the parameters, the cost function does the real job, right? If it is correct, what does cost function do in ...
1
vote
1
answer
209
views
Finding sequence combinations that impact target variable the most
One can create a time series model to predict a target variable. What I need to do is find the input combinations and sequences that impact the target variable the most. In this case, the input data ...
CommunityBot
- 1
1
vote
1
answer
209
views
issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory
I am trying to load the ckpt file and getting error
PytorchStreamReader failed reading zip archive: failed finding central directory
Here is the code
...
- 146
1
vote
1
answer
525
views
Training neural network for regression with gaussian output layer
How does one train a neural network model that does regression over real values, using a gaussian output layer? ie estimating the mean and std parameters of the prediction. Since during training there ...
CommunityBot
- 1
3
votes
2
answers
681
views
LeNet-5 - combining feature maps in C3 layer
Famous LeNet-5 architecture looks like this:
The output of layer S2 has dimension: 10x10x6 - so basically an image with 6 convultions applied to it to derive features.
If each dimension was again ...
CommunityBot
- 1
0
votes
2
answers
88
views
Creating dataset - imbalanced or balanced?
I'm trying to make an image classification model and I have 5 classes - A, B, C, D, E. The goal is to get the highest possible classification accuracy.
I have a database of images and I'm selecting ...
CommunityBot
- 1
0
votes
2
answers
131
views
Is the Cross Entropy Loss important at all, because at Backpropagation only the Softmax probability and the one hot vector are relevant?
Is the Cross Entropy Loss (CEL) important at all, because at Backpropagation (BP) only the Softmax (SM) probability and the one hot vector are relevant?
When applying BP, the derivative of CEL is the ...
CommunityBot
- 1
0
votes
1
answer
185
views
Image classification with CNN Python
I'm working on image classification using CNN, my dataset contains more than 50 classes (50 folders) which represent the types of car parts, and in each folder we have vehicle brands, each vehicle ...
CommunityBot
- 1
2
votes
1
answer
366
views
How to get vector representations(or embeddings) of time series?
Even if a time series is constructed up of numbers only, finding abstract fixed-dim vector representation would be interesting for classification/clustering purposes. As we can learn & find ...
CommunityBot
- 1
2
votes
1
answer
430
views
Cat2Vec implementation X = categorical and y = categorical
I am trying to convert categorical values (zipcodes) with Cat2Vec into a matrix which can be used as an input shape for categorical prediction of a target with binary values.
After reading several ...
CommunityBot
- 1
0
votes
1
answer
460
views
Tensorflow model works for classification but not for regression (all predictions equal the output layer bias)
I'm trying to build a model for FX prediction. It's giving some promising results for classifying each period as buy/sell/neutral. When used as a classifier, actual returns are converted to 0, 1, or ...
CommunityBot
- 1
4
votes
4
answers
661
views
Where does the "deep learning needs big data" rule come from
When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
CommunityBot
- 1
0
votes
0
answers
22
views
Model Performance not improving
I am currently working with a GNN (a Graph attention Model) based model and the main task is to do Graph prediction. My model doesnot improve its performance when I change the number of heads or the ...
- 1
0
votes
1
answer
80
views
Is it possible to implement a vectorized version of a Maxout activation function?
I want to implement an efficient and vectorized Maxout activation function using python numpy.
Here is the paper in which "Maxout Network" was introduced (by Goodfellow et al).
For example, ...
CommunityBot
- 1
0
votes
0
answers
33
views
RetNet Paper Multi Scale Retention dimemsion question
From the paper:
https://arxiv.org/pdf/2307.08621.pdf
But since X is of size n by $d_{model}$.
How can we compute $XW_Q$? Since the row length of X which is $d_{model}$ is not the same as the column ...
1
vote
1
answer
106
views
Train on multi-domains, then fine-tune on specific domain
Would it make sense to first train a model on images from multiple domains, and then do "fine-tuning" on one specific domain to improve its performance on it?
For instance, one could train an object ...
CommunityBot
- 1
2
votes
1
answer
2k
views
Backpropagation of convolutional neural network - confusion
I've already seen many articles about this topic and Backpropagation In Convolutional Neural Networks by Jefkine seems to be the best. Although, as author said,
For the purposes of simplicity we ...
CommunityBot
- 1
1
vote
1
answer
1k
views
Classification of scanned documents in pdf files using deep learning or NLP
I know classifying images using cnn but I have a problem where I have multiple types of scanned documents in a pdf file on different pages. Some types of scanned documents present in multiple pages ...
CommunityBot
- 1
1
vote
1
answer
25
views
A good set of datasets/models for testing an NLP technique
I am a machine learning researcher who up until this point has primarily worked on Computer Vision problems. However, I have an idea for an NLP technique involving a novel Transformer architecture, ...
- 106
1
vote
2
answers
98
views
Dynamically remove data from training dataset
I was wondering today if it would be a good approach to remove data dynamically from the training dataset when learning a neural network. Assuming a classification task, the approach would be ...
CommunityBot
- 1
4
votes
1
answer
276
views
Importance/intuition behind stacking RNNs
Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice.
What's the intuition ...
CommunityBot
- 1
2
votes
1
answer
147
views
Ways to share Pytorch model without revealing architecture?
We are trying to give a model to collaborators but would like to protect the IP. What are some ways to encrypt/hide/compile the definition when sharing a trained model?
CommunityBot
- 1
0
votes
0
answers
26
views
Using Embedding For Regularization
Is using embeddings for regularization a valid practice?
My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
0
votes
1
answer
93
views
A way to init sentence embedding for unsupervised text clustering, better than glove wordvec?
For unsupervised text clustering, the key thing is the init embedding for text.
If we want to use deepcluster for text, the problem for text is how to get the init embedding from deep model.
BERT can ...
CommunityBot
- 1
3
votes
2
answers
2k
views
Cross validation for convolutional neural network
I am using Keras to create a CNN model, and I would to use K-fold cross-validation to train the dataset.
The dataset contains images and I am using ...
CommunityBot
- 1
2
votes
1
answer
186
views
Why yolo4 pytorch re-training loss seems high as like first time training?
I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git.
I generated checkpoints by giving ...
CommunityBot
- 1
0
votes
1
answer
864
views
How to reduce RMS error value in regression analysis & predictions - feature engineering, model selection
There's this dataset containing the metadata of Twitch's top 1,000 streamers of 2020. You can have the details here.
I am currently participating in a challenge to predict the values for Followers ...
CommunityBot
- 1