Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,147 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
1answer
103 views

What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small.I have 12K sentences and only 10% of them are from positive ...
6
votes
1answer
84 views

how to generate automatically images meshing up different shapes with a deep learning software?

My pursuite is to generate something like a grottesque(a kind of painting producing human-animals and plants hybrids). I need to do something like this paints in order to create an art exhibition. I ...
5
votes
1answer
86 views

How to train the predicting boxes in a YOLO network?

I have just finished this tutorial that explains how YOLO networks work. Instead of training the network's weights with a training set, the author loads pre-trained weights and uses them to test the ...
5
votes
1answer
2k views

Gumbel-Softmax trick vs Softmax with temperature

From what I understand, the Gumbel-Softmax trick is a technique that enables us to sample discrete random variables, in a way that is differentiable (and therefore suited for end-to-end deep learning)....
5
votes
0answers
1k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
5
votes
1answer
128 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
1answer
2k views

Recurrent neural network multiple types of input Keras

For a project I want to use recurrent neural networks, however my knowledge on this subject is still somewhat limited. I do have some experience with convolutional nets and traditional neural networks....
4
votes
1answer
757 views

Autocomplete with deep learning

I got interested in autocompletion using deep learning and tutorials that I found where conditioned always on specific number of characters (given 40 characters predict the next character or the whole ...
4
votes
0answers
92 views

What is the relationship between “landmark Detection” and “landmark localization”

I am reading this paper "Grand Challenge of 106-Point Facial Landmark Localization" In the context of face recognition "Landmark Detection" is to detect a face by matching landmarks on a face. "...
4
votes
1answer
80 views

How to train neural word embeddings?

So I am new to Deep Learning and NLP. I have read several blog posts on medium, towardsdatascience and papers where they talk about pre-training the word embeddings in an unsupervised fashion and then ...
4
votes
1answer
32 views

Adding more emphasis on most recent data in CNNs

I am using a CNN for multivariate time series analysis. The input size is (batch_size, 500, 30) i.e 30 variables and 500 time steps. I would like to put more ...
4
votes
2answers
444 views

issue with early-stopping on f1 score with imbalanced data

I have a highly imbalanced dataset with less than 0.5% of the minor class. Using Keras, I'm training DNN on the training set and evaluate performance on validation set. Loss function is ...
4
votes
1answer
733 views

Implementing spatio-temporal convolutions in pytorch

I am trying to implement a layer to perform the (2+1)D convolutions described in this paper: https://arxiv.org/pdf/1711.11248.pdf The basic idea is as follows: Let's say I have a 3D convolutional ...
4
votes
0answers
253 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
2k views

Gradient flow through concatenation operation

I need help in understanding the gradient flow through a concatenation operation. I'm implementing a network (mostly a CNN) which has a concatenation operation (in pytorch). The network is defined ...
4
votes
1answer
1k views

Convolutional Network for Text Classification

I am trying to train a convolutional neural network with Keras at recognizing tags for Stack Exchange questions about cooking. The i-th question element of my data-set is like this: ...
4
votes
0answers
254 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
0answers
119 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
4
votes
1answer
60 views

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
3
votes
0answers
87 views

Chess deep learning siamese network overfitting when shouldn't in theory

I'm trying to implement a chess deep learning model like shown in the paper "DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess" (https://www.cs.tau.ac.il/~wolf/papers/deepchess....
3
votes
1answer
51 views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
3
votes
1answer
29 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
3
votes
0answers
23 views

Keras model with second to last sigmoid activated Conv1D layer followed by globalMaxPool outputs values outside [0,1]. Why?

I am trying to train a binary classifier. It is a residual network with skip layers etc. but ultimately, the bottom two layers are a 1D convolution with sigmoid activation followed by a global max ...
3
votes
1answer
355 views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
3
votes
0answers
275 views

How to use PCA in CNN for image recognition using Keras?

I created a CNN model for image classification and I want to use Principal Component Analysis (PCA) but when I run pca.fit() code, the code still running for hours ...
3
votes
0answers
103 views

Why categorical cross entropy loss is not correlated with NLP scores?

I'm training a deep network for image captioning which is consist of one CNN and three GRUs. During training epoch by epoch model loss (categorical cross entropy) decreases but when I'm measuring <...
3
votes
0answers
34 views

A Deep CNN model delivering better results with standardization, when compared with normalization

I developed a deep CNN model, based on the architecture discussed in this paper, to generate predictions for time series data. My training data is shown in the figure below: In order to train the ...
3
votes
0answers
155 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
2answers
2k views

GAN - am I seeing mode collapse? Common fixes not working

I have a 2 part question. Context I am learning about GANs and writing my own starting from the very simplest example of adversarial learning (1-parameter node), then implementing a very simple 1-...
3
votes
0answers
229 views

How to reload cell in jupyter notebook? After OSError: [Errno 12] Cannot allocate memory

I got an error: OSError: [Errno 12] Cannot allocate memory I deleted some files. And I have free memory. I don't want start learning again from first epoch. (I got ...
3
votes
1answer
207 views

Understanding Youtube recommender (candidate generation step)

I'm trying to understand https://storage.googleapis.com/pub-tools-public-publication-data/pdf/45530.pdf Their candidate generation step outputs top N items via softmax (with negative sampling) at ...
3
votes
0answers
202 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
1answer
551 views

Different learning rate for each of the layers?

I noticed that some popular deep learning frameworks like Keras or Pytorch allow you to set different learning rate for each layer. What are the benefits of that approach?
3
votes
0answers
151 views

Training deep CNN with noisy dataset

I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering. The train set captures ...
3
votes
1answer
222 views

Dataset image size and inference speed

Does training/fine tuning a pre-trained model on a the same dataset but with sizes scaled down (e.g., by 70%) improve inference speed? More generally, does training a CNN on smaller images improve ...
3
votes
1answer
310 views

Comparison between addition and multiplication function in deep neural network?

I designed a specific Convolution Neural Network to study in the area of image processing. The network has a part that there are two tensors which have to be transformed into a tensor in order to be ...
3
votes
1answer
112 views

Can a neural network recognize a letter B as an A if your trained it so?

You have a neural network. And you have, say, pictures of $100,000$ hand-written letters (A-Z). Now you make a typical Training and the neural network will recognize an A as an A, a B as a B, ... Now ...
3
votes
2answers
2k views

What is the best way to read SQL dataset in to Tensorflow?

What is the best way to read SQL database in to Tensorflow? Currently, I am using Postgres on server and developed DL algorithm on Tensorflow using Jupyter Lab. How can I import data into Jupyter Lab ...
3
votes
0answers
2k views

ValueError: Numpy arrays that you are passing to your model is not the size the model expected

I am trying to perform concatenation on the Bidirectinal LSTM layer. I have my model defined like this: ...
3
votes
0answers
100 views

How many pairs of image needed for training Siamese Network? And how to augment them?

Hi Deep Learning researchers and engineers, Does anyone have experience in Siamese Network regarding the training data size? How many pairs of image do I need to train a Siamese Network? And what ...
3
votes
0answers
810 views

Cross validation for convolutional neural network

I am using Keras to create a CNN model, and I would to use K-fold cross-validation to train the dataset. The dataset contains images and I am using ...
3
votes
0answers
250 views

convolutional neural network with cross validation in Keras

I want to use K-fold cross-validation on my dataset of images. I am reading the data (images) from a directory. How do I use cross validation with convolutional neural network in Keras?
3
votes
0answers
891 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
3
votes
1answer
192 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...
3
votes
1answer
552 views

Does Pooling remove spatial information of image in CNN?

Pr. Geoffrey Hinton has pointed out that pooling-layers remove spatial feature information. But, essentially, does the process that last convolutional layer's features are flattened for FC layer makes ...
3
votes
1answer
40 views

Is there any work done on reconfigurable convolutional neural networks?

Convolutional Neural networks are used in supervised learning meaning models are always "set in stone" after training (architecture and paramters) so this might not even be possible, but is there any ...
3
votes
1answer
80 views

Is it a red flag that increasing the number of parameters makes the model less able to overfit small amounts of data?

I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~...
3
votes
1answer
1k views

Overfitting in CNN

I am training a VGG net on STL-10 dataset I am getting Top-5 validation accuracy about 98% and Top-1 validation accuracy about 83% But both the Top-1 and Top-5 Training accuracy is reaching 100% ...
3
votes
0answers
101 views

Deep advantage learning: how to predict the value

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. ...
3
votes
0answers
176 views

Identifying computer scanned digits

I have digit images as below which I would like to identify: Some are of slightly worse quality : The images are not of a fixed resolution but are mostly in the range (80*20 to 130 *40). Due to ...

1
2 3 4 5
23