Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,057 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8
votes
0answers
327 views

Differences between gradient calculated by different reduction methods in PyTorch

I'm playing with different reduction methods provided in built-in loss functions. In particular, I would like to compare the following. The averaged gradient by performing backward pass for each loss ...
6
votes
1answer
78 views

how to generate automatically images meshing up different shapes with a deep learning software?

My pursuite is to generate something like a grottesque(a kind of painting producing human-animals and plants hybrids). I need to do something like this paints in order to create an art exhibition. I ...
6
votes
1answer
125 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
1answer
43 views

How to train the predicting boxes in a YOLO network?

I have just finished this tutorial that explains how YOLO networks work. Instead of training the network's weights with a training set, the author loads pre-trained weights and uses them to test the ...
5
votes
0answers
1k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
5
votes
1answer
2k views

Recurrent neural network multiple types of input Keras

For a project I want to use recurrent neural networks, however my knowledge on this subject is still somewhat limited. I do have some experience with convolutional nets and traditional neural networks....
4
votes
0answers
27 views

Detecting abundance of a certain periodic pattern in a time series?

I am really stumped at the moment about how to solve a particular problem. I have many time series like this: This represents the number of hours a person spends on a website each day throughout the ...
4
votes
0answers
62 views

What is the relationship between “landmark Detection” and “landmark localization”

I am reading this paper "Grand Challenge of 106-Point Facial Landmark Localization" In the context of face recognition "Landmark Detection" is to detect a face by matching landmarks on a face. "...
4
votes
1answer
72 views

How to train neural word embeddings?

So I am new to Deep Learning and NLP. I have read several blog posts on medium, towardsdatascience and papers where they talk about pre-training the word embeddings in an unsupervised fashion and then ...
4
votes
1answer
27 views

Adding more emphasis on most recent data in CNNs

I am using a CNN for multivariate time series analysis. The input size is (batch_size, 500, 30) i.e 30 variables and 500 time steps. I would like to put more ...
4
votes
1answer
318 views

issue with early-stopping on f1 score with imbalanced data

I have a highly imbalanced dataset with less than 0.5% of the minor class. Using Keras, I'm training DNN on the training set and evaluate performance on validation set. Loss function is ...
4
votes
0answers
235 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
1k views

Gradient flow through concatenation operation

I need help in understanding the gradient flow through a concatenation operation. I'm implementing a network (mostly a CNN) which has a concatenation operation (in pytorch). The network is defined ...
4
votes
1answer
1k views

Convolutional Network for Text Classification

I am trying to train a convolutional neural network with Keras at recognizing tags for Stack Exchange questions about cooking. The i-th question element of my data-set is like this: ...
4
votes
0answers
247 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
0answers
114 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
3
votes
0answers
131 views

How to use PCA in CNN for image recognition using Keras?

I created a CNN model for image classification and I want to use Principal Component Analysis (PCA) but when I run pca.fit() code, the code still running for hours ...
3
votes
0answers
89 views

Why categorical cross entropy loss is not correlated with NLP scores?

I'm training a deep network for image captioning which is consist of one CNN and three GRUs. During training epoch by epoch model loss (categorical cross entropy) decreases but when I'm measuring <...
3
votes
0answers
28 views

A Deep CNN model delivering better results with standardization, when compared with normalization

I developed a deep CNN model, based on the architecture discussed in this paper, to generate predictions for time series data. My training data is shown in the figure below: In order to train the ...
3
votes
0answers
112 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
0answers
162 views

How to reload cell in jupyter notebook? After OSError: [Errno 12] Cannot allocate memory

I got an error: OSError: [Errno 12] Cannot allocate memory I deleted some files. And I have free memory. I don't want start learning again from first epoch. (I got ...
3
votes
0answers
172 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
1answer
405 views

Different learning rate for each of the layers?

I noticed that some popular deep learning frameworks like Keras or Pytorch allow you to set different learning rate for each layer. What are the benefits of that approach?
3
votes
0answers
130 views

Training deep CNN with noisy dataset

I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering. The train set captures ...
3
votes
1answer
186 views

Dataset image size and inference speed

Does training/fine tuning a pre-trained model on a the same dataset but with sizes scaled down (e.g., by 70%) improve inference speed? More generally, does training a CNN on smaller images improve ...
3
votes
1answer
238 views

Why do most GAN (Generative Adversarial Network) implementations have symmetric discriminator and generator architectures?

For example, if the discriminator is a vanilla network of n layers, each with n(i) units, then, typically, the generator will also be a vanilla network of n layers, each with n(n-i) units (except the ...
3
votes
1answer
1k views

What is the best way to read SQL dataset in to Tensorflow?

What is the best way to read SQL database in to Tensorflow? Currently, I am using Postgres on server and developed DL algorithm on Tensorflow using Jupyter Lab. How can I import data into Jupyter Lab ...
3
votes
0answers
900 views

ValueError: Numpy arrays that you are passing to your model is not the size the model expected

I am trying to perform concatenation on the Bidirectinal LSTM layer. I have my model defined like this: ...
3
votes
0answers
77 views

How many pairs of image needed for training Siamese Network? And how to augment them?

Hi Deep Learning researchers and engineers, Does anyone have experience in Siamese Network regarding the training data size? How many pairs of image do I need to train a Siamese Network? And what ...
3
votes
0answers
728 views

Cross validation for convolutional neural network

I am using Keras to create a CNN model, and I would to use K-fold cross-validation to train the dataset. The dataset contains images and I am using ...
3
votes
1answer
1k views

What does SpatialDropout1D() do to output of Embedding() in Keras?

Keras model looks like this ...
3
votes
0answers
216 views

convolutional neural network with cross validation in Keras

I want to use K-fold cross-validation on my dataset of images. I am reading the data (images) from a directory. How do I use cross validation with convolutional neural network in Keras?
3
votes
0answers
827 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
3
votes
1answer
156 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...
3
votes
0answers
39 views

Fully connected layer output explodes, but weights, gradients, and inputs all have sane values

I'm trying to train a GAN, and the architecture includes a fully connected layer before the output activation function. In my case, by the second training iteration this layer's output always explodes....
3
votes
0answers
44 views
3
votes
1answer
452 views

Does Pooling remove spatial information of image in CNN?

Pr. Geoffrey Hinton has pointed out that pooling-layers remove spatial feature information. But, essentially, does the process that last convolutional layer's features are flattened for FC layer makes ...
3
votes
1answer
76 views

Is it a red flag that increasing the number of parameters makes the model less able to overfit small amounts of data?

I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~...
3
votes
1answer
922 views

Overfitting in CNN

I am training a VGG net on STL-10 dataset I am getting Top-5 validation accuracy about 98% and Top-1 validation accuracy about 83% But both the Top-1 and Top-5 Training accuracy is reaching 100% ...
3
votes
1answer
34 views

alternatives to regression to decide weights in an expression

I have a use case in which I am required to predict variable y which depends on 5 variables, xi. Consider something like [ w1*x1 + w2*x2 + w3*x3 + w4*x4 + w5*x5 = y] This expression doesn't ...
3
votes
0answers
88 views

Deep advantage learning: how to predict the value

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. ...
3
votes
0answers
158 views

Identifying computer scanned digits

I have digit images as below which I would like to identify: Some are of slightly worse quality : The images are not of a fixed resolution but are mostly in the range (80*20 to 130 *40). Due to ...
3
votes
0answers
510 views

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'(Stateful LSTM Keras)

So I have a trained LSTM model with which I am trying to predict future values. The model is stateful as seen below ...
3
votes
0answers
447 views

Keras - Masking CNNs

I have a 3D tensor on which I apply 2D convolutions. Sometimes, this 3D is padded both in width and height to have a fixed size. How could I apply masking (like with RNNs) so that the gradients ...
3
votes
1answer
409 views

Multi-task learning for Multi-label classification?

I have a multi-label classification problem wherein each example can belong to one of the pre-defined classes (or can belong to none of them). I was wondering if I can somehow apply multi-task ...
3
votes
0answers
294 views

Autoencoder behavior with All White/Black MNIST

I am using a stock auto-encoder anomaly detector from Deeplearning4j. I was getting unexpected results from my own variant of the auto-encoder, which looks for anomalies in my own (non-image) data, ...
3
votes
0answers
782 views

Help with the following error: Variable already exists, disallowed. Did you mean to set reuse=True in VarScope?

I am not sure how to handle this error. This is from an RNN tutorial found here. I vaguely understand that the variables need to be able to be reused, but I don't know how to implement this fix. ...
3
votes
0answers
365 views

Using an autoencoder to mimic independent component analysis?

I'm trying to use autoencoders in keras to create a linear transformation similar to independent component analysis (ICA) (using this to denoise electroencephalographic data, time series of 64x100000 ...
3
votes
2answers
242 views

How to create domain rules from raw unstructured text using NLP and deep learning?

How to create domain rules from raw unstructured text using NLP and deep learning techniques ? For example for the below text on symptoms of Dengue, all three look pretty similar but if you want to ...
3
votes
0answers
807 views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...