Skip to main content

Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
0 votes
1 answer
50 views

How to categorize unlabelled promotional email data

I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
0 votes
1 answer
356 views

predict_classes() returning only 0 or 1 for multiclass image classification

I am trying to build a multi class image classifier but the only returns 0 or 1 . Why is it not returning "Rock" , "Paper" , "Scissor" ? and why only 0 and 1 but not 2? CODE: ...
4 votes
4 answers
728 views

Where does the "deep learning needs big data" rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
1 vote
1 answer
319 views

LSTM with input of actual time step

I'm working on an implementation of LSTM neural network to forecast energy consumption. I have a dataset with load, series of weather parameters and indicator of it's bank holiday or not. I first ...
28 votes
3 answers
13k views

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
0 votes
0 answers
10 views

Should multiple categorical embeddings be combined for a conditional GAN (cGAN)?

I'm trying to make a conditional GAN (cGAN) that generates YouTube thumbnails based on a title and a video category/genre. It's not working whatsoever, not even close, and so I'm trying to go back to ...
1 vote
2 answers
511 views

Negative examples for a Yes/No image classification neural network

I am trying to retrain a neural network using transfer learning that can classify whether an image has a certain object, say, a car. My positive sample dataset is quite small, only 2500~ images. It ...
0 votes
0 answers
10 views

Courses or lectures or books on machine learning or AI in general that have a lot of theoretical and practical mathematics but also practical coding

I'm looking for deep learning, or machine learning more generally, or artificial intelligence more generally, courses or lectures or books, that have a lot of theoretical and practical mathematics but ...
0 votes
2 answers
440 views

Is Siamese network rotation invariant?

Is Siamese network rotation invariant which means if I train my siamese network on the different rotated versions of the same image so will it treat each image as different image or same. Also if I ...
1 vote
1 answer
180 views

Should deep layers ever have more units than the input layer?

i.e. if a model with 10 inputs, say, ...
2 votes
2 answers
132 views

Why does a filter need to be applied to the output of the input gate before cell state is added to?

In a neural network there are 4 gates: input, output, forget and a gate whose output performs element wise multiplication with the output of the input gate, which is added to the cell state (I don't ...
2 votes
1 answer
458 views

Cat2Vec implementation X = categorical and y = categorical

I am trying to convert categorical values (zipcodes) with Cat2Vec into a matrix which can be used as an input shape for categorical prediction of a target with binary values. After reading several ...
2 votes
1 answer
2k views

Backpropagation of convolutional neural network - confusion

I've already seen many articles about this topic and Backpropagation In Convolutional Neural Networks by Jefkine seems to be the best. Although, as author said, For the purposes of simplicity we ...
0 votes
2 answers
348 views

Self-Attention Summation and Loss of Information

In self-attention, the attention for a word is calculated as: $$ A(q, K, V) = \sum_{i} \frac{exp(q.k^{<i>})}{\sum_{j} exp(q.k^{<j>})}v^{<i>} $$ My question is why we sum over the ...
1 vote
1 answer
16 views

Is it appropriate to use KL Divergence as a loss function for a 1x3 regression model?

I have a regression model with a 1x3 output, which means it predicts three continuous values. I'm wondering if it would be appropriate to use the Kullback-Leibler (KL) Divergence as the loss function ...
0 votes
2 answers
150 views

Fast AI Lesson 4 - MNIST. Confused about multiplying weights by pixels?

I’m on lesson 4 of the Fast AI "Deep Learning for Coders" course, and have been back through the same lesson a few times now but I don’t think I’m quite getting a few things. I want to have ...
0 votes
2 answers
92 views

Batch Normalization vs Layer Normalization

In Batch Normalization, mean and standard deviation are calculated feature wise and normalization step is done instance wise and in Layer Normalization mean and standard deviation are calculated ...
1 vote
1 answer
54 views

How to define similarity between nodes in original graph?

While there has been a lot of talk about defining the similarity between nodes in the embedding space, I don't seem to come across any talking about defining the similarity between nodes in the ...
1 vote
1 answer
69 views

N-gram based Language Models learned using an Encoder-Decoder Model

I have been going through a N-gram based Language Model learned using an Encoder-Decoder Model for Email smart compose. The program outputs only one prediction for the given input. I want to know how ...
6 votes
2 answers
528 views

How do regression loss functions like MAE and MSE work although they remove the plus/minus sign?

I have a question about regression loss functions like Mean Absolute Error (MAE) and Mean Squared Error (MSE) used in deep learning. When we calculate these losses, we remove the plus/minus sign from ...
1 vote
2 answers
1k views

Vanishing gradient problem even after existence of ReLu function?

Let's say I have a deep neural network with 50 hidden layers and at each neuron of hidden layer the ReLu activation function is used. My question is Is it possible for vanishing gradient problem to ...
1 vote
1 answer
819 views

Training neural network for regression with gaussian output layer

How does one train a neural network model that does regression over real values, using a gaussian output layer? ie estimating the mean and std parameters of the prediction. Since during training there ...
0 votes
1 answer
176 views

Train and Validation Curve

I'm new in DeepLearning. I'm not good at understanding and commenting on graphics.Can you help me with these graphs
0 votes
1 answer
99 views

A way to init sentence embedding for unsupervised text clustering, better than glove wordvec?

For unsupervised text clustering, the key thing is the init embedding for text. If we want to use deepcluster for text, the problem for text is how to get the init embedding from deep model. BERT can ...
0 votes
1 answer
21 views

Interpretation of PPO learning curve, value loss, policy loss

my PPO training for a custom gymnasium environment resulted in following outcome. I would need some advice how to interpret the results and where to start activities to improve. Thank you very much ...
0 votes
0 answers
26 views
+50

How specific should I be with my region of interest in image data for training a CNN model for better accuracies?

I am trying to train a 3D CNN model for classification of cancer stages on a dataset that comprises of head to neck CT image series which is split into 5 classes corresponding to the stages of cancer....
0 votes
1 answer
214 views

How many bounding boxes does the YOLOv6 model predict in total before thresholding?

I understand that the YOLOv5 model predicts 25200 bounding boxes between all 3 levels of output. How many does the YOLOv6 model predict, if the input resolution is 640x640?
0 votes
0 answers
9 views

High Validation Accuracy + High Loss Score and High Training Accuracy + Low Loss Score?

I am having a wierd observation in my experiments, I am using BERT with adapter and lora PEFT methods for domain adaptation. I first trained the adapter on Unlabled target domain dataset using MLM, ...
2 votes
2 answers
309 views

Dynamically remove data from training dataset

I was wondering today if it would be a good approach to remove data dynamically from the training dataset when learning a neural network. Assuming a classification task, the approach would be ...
0 votes
1 answer
16 views

What is the "fast version" of ZFNet referenced in SPPNet and Faster R-CNN papers?

I'm reading old papers: SPPNet: Link Faster R-CNN: Link In both cases, the authors refer to a "fast version of Zeiler and Fergus (ZF) Net"; specifically: In SPPNet: ZF-5: this ...
0 votes
1 answer
49 views

Classification of sequential data

I'm currently trying to classify discrete sequential data into five classes with machine learning. The setup is the following: The actual object is filled with various properties, but to separate the ...
1 vote
1 answer
117 views

Working Behavior of BERT vs Transformers vs Self-Attention+LSTM vs Attention+LSTM on the scientific STEM data classification task?

So I just used BERT pre-trained with Focal Loss to classify Physics, Chemistry, Biology and Mathematics and got a good f-1 macro of 0.91. It is good given it only had to look for the tokens like ...
0 votes
0 answers
7 views

Binary Cross Entropy Loss Not Learning

I am trying to implement a CNN that receives as input a 640 by 368 sized tensor where only the center 30 columns (i.e. columns 169 to 199) keep their real values and everything else is set to 0 and ...
1 vote
0 answers
36 views

Why can't I replicate the results from this paper?

I'm trying to train a model to evaluate chess positions, following the methodology from this paper (note that the author presents several different architectures, but I'm only looking at the ANN with ...
3 votes
1 answer
124 views

Perceptron Learning Rule

I am new to Machine Learning and Data Science. By spending some time online, I was able to understand the perceptron learning rule fairly well. But I am still clueless about how to apply it to a set ...
15 votes
2 answers
13k views

Why should we use (or not) dropout on the input layer?

People generally avoid using dropout at the input layer itself. But wouldn't it be better to use it? Adding dropout (given that it's randomized it will probably end up acting like another regularizer)...
5 votes
2 answers
670 views

Loss for ordered multi class data in classification

Assume data which is labeled $y_i \in \left\{ 1, 2, 3, \ldots, 9, 10\right\}$. Assume the labels are ordered, namely, given $y_i = 10$ to estimate $\hat{y}_{i} = 1$ is much worse than $\hat{y}_{i} = ...
3 votes
2 answers
760 views

LeNet-5 - combining feature maps in C3 layer

Famous LeNet-5 architecture looks like this: The output of layer S2 has dimension: 10x10x6 - so basically an image with 6 convultions applied to it to derive features. If each dimension was again ...
0 votes
1 answer
293 views

Prediction using words which were not in training in a CNN with pre-trained word embeddings

In sentence classification using pre-trained embeddings(fasttext) in a CNN, how does the CNN predict the category of a sentence when the words were not in the training set? I think the trained model ...
3 votes
1 answer
779 views

Discriminator of a Conditional GAN with continuous labels

OK, let's say we have well-labeled images with non-discrete labels such as brightness or size or something and we want to generate images based on it. If it were done with a discrete label it could ...
0 votes
1 answer
210 views

Image classification with CNN Python

I'm working on image classification using CNN, my dataset contains more than 50 classes (50 folders) which represent the types of car parts, and in each folder we have vehicle brands, each vehicle ...
0 votes
0 answers
7 views

Exploding Gradients when using Momentum!

I'm implementing gradient descent with momentum from scratch in NumPy. Issue is that I'm receiving exploding gradients after a couple of epochs. I could apply L2 regularization, which seems to work ...
1 vote
1 answer
71 views

Autoencoder not learning walk forward image transformation

I have a series of 15 frames with (60 rows x 50 columns). Over the course of those 15 frames, the moon moves from the top left to the bottom right. Data = https://github.com/aiqc/AIQC/tree/main/...
1 vote
1 answer
49 views

wierd neural network approache

I'm working on a problem where I need to create a neural network to optimize the seating arrangement for 24 unique individuals in a 6x4 grid, minimizing conflicts between adjacent (up,down,left,right) ...
0 votes
1 answer
126 views

different range of target values in neural network

I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100). The target ...
0 votes
0 answers
8 views

Take a nap (SLEEP) at the end of each epoch

I wonder if that is a training strategy that if we could take a SLEEP() function at the end of each epoch. ...
1 vote
1 answer
275 views

Calculate importance of input data bands for CNN image classification?

I constructed and trained a convolutional neural network using Keras in R with the TensorFlow backend. I feed the network with multispectral images for a simple image classification. Is there some way ...
0 votes
1 answer
2k views

Epoch 1/5 won't stop

When i run my code with 5 epochs, code gets stuck at first epoch and run continuesly. I tried applying various parameters but couldn't make it. here is my code... ...
0 votes
1 answer
1k views

How to reduce RMS error value in regression analysis & predictions - feature engineering, model selection

There's this dataset containing the metadata of Twitch's top 1,000 streamers of 2020. You can have the details here. I am currently participating in a challenge to predict the values for Followers ...
12 votes
1 answer
21k views

Keras LSTM with 1D time series

I'm learning how to use Keras and I've had reasonable success with my labelled dataset using the examples on Chollet's Deep Learning for Python. The data set is ~1000 Time Series with length 3125 with ...

1
2 3 4 5
98