Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,267 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
1answer
134 views

What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small.I have 12K sentences and only 10% of them are from positive ...
6
votes
2answers
3k views

Gumbel-Softmax trick vs Softmax with temperature

From what I understand, the Gumbel-Softmax trick is a technique that enables us to sample discrete random variables, in a way that is differentiable (and therefore suited for end-to-end deep learning)....
6
votes
2answers
94 views

how to generate automatically images meshing up different shapes with a deep learning software?

My pursuite is to generate something like a grottesque(a kind of painting producing human-animals and plants hybrids). I need to do something like this paints in order to create an art exhibition. I ...
5
votes
0answers
1k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
5
votes
1answer
129 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
1answer
2k views

Recurrent neural network multiple types of input Keras

For a project I want to use recurrent neural networks, however my knowledge on this subject is still somewhat limited. I do have some experience with convolutional nets and traditional neural networks....
4
votes
0answers
257 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My programme gives the following error message: ...
4
votes
0answers
98 views

What is the relationship between “landmark Detection” and “landmark localization”

I am reading this paper "Grand Challenge of 106-Point Facial Landmark Localization" In the context of face recognition "Landmark Detection" is to detect a face by matching landmarks on a face. "...
4
votes
2answers
2k views

GAN - am I seeing mode collapse? Common fixes not working

I have a 2 part question. Context I am learning about GANs and writing my own starting from the very simplest example of adversarial learning (1-parameter node), then implementing a very simple 1-...
4
votes
1answer
81 views

How to train neural word embeddings?

So I am new to Deep Learning and NLP. I have read several blog posts on medium, towardsdatascience and papers where they talk about pre-training the word embeddings in an unsupervised fashion and then ...
4
votes
1answer
32 views

Adding more emphasis on most recent data in CNNs

I am using a CNN for multivariate time series analysis. The input size is (batch_size, 500, 30) i.e 30 variables and 500 time steps. I would like to put more ...
4
votes
2answers
600 views

issue with early-stopping on f1 score with imbalanced data

I have a highly imbalanced dataset with less than 0.5% of the minor class. Using Keras, I'm training DNN on the training set and evaluate performance on validation set. Loss function is ...
4
votes
1answer
787 views

Implementing spatio-temporal convolutions in pytorch

I am trying to implement a layer to perform the (2+1)D convolutions described in this paper: https://arxiv.org/pdf/1711.11248.pdf The basic idea is as follows: Let's say I have a 3D convolutional ...
4
votes
0answers
1k views

How to draw a simple LSTM network

I'm new to deep learning, I am learning LSTM for my PhD work. This is a simple LSTM network for sequence classification. This code is from MATLAB tutorial: ...
4
votes
0answers
262 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
149 views

How do we pass data to a RNN?

Let's say we have A1, A2, ... , Am different articles in the corpus and each of them has W1, W2, ....., Ww words. We are training a language model on them. Do we: Scheme 1 Take the first batch of ...
4
votes
1answer
2k views

Gradient flow through concatenation operation

I need help in understanding the gradient flow through a concatenation operation. I'm implementing a network (mostly a CNN) which has a concatenation operation (in pytorch). The network is defined ...
4
votes
1answer
1k views

Convolutional Network for Text Classification

I am trying to train a convolutional neural network with Keras at recognizing tags for Stack Exchange questions about cooking. The i-th question element of my data-set is like this: ...
4
votes
0answers
256 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
1answer
128 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
4
votes
1answer
138 views

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having ...
3
votes
0answers
27 views

Robustness of hyperparameter tuning

I use a Bayesian hyperparameter (HP) optimization approach (BOHB) to tune a deep learning model. However, the resulting model is not robust when repeatedly applied to the same data. I know, I could ...
3
votes
0answers
31 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i....
3
votes
0answers
112 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
3
votes
1answer
116 views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
3
votes
1answer
30 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
3
votes
0answers
848 views

SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors

I am writing Encoder-Decoder architecture with Bahdanau Attention using tf.keras with TensorFlow 2.0. Below is my code This is working with TensorFlow 1.15 but getting the error in 2.0. you can check ...
3
votes
0answers
26 views

Keras model with second to last sigmoid activated Conv1D layer followed by globalMaxPool outputs values outside [0,1]. Why?

I am trying to train a binary classifier. It is a residual network with skip layers etc. but ultimately, the bottom two layers are a 1D convolution with sigmoid activation followed by a global max ...
3
votes
1answer
447 views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
3
votes
0answers
344 views

How to use PCA in CNN for image recognition using Keras?

I created a CNN model for image classification and I want to use Principal Component Analysis (PCA) but when I run pca.fit() code, the code still running for hours ...
3
votes
0answers
109 views

Why categorical cross entropy loss is not correlated with NLP scores?

I'm training a deep network for image captioning which is consist of one CNN and three GRUs. During training epoch by epoch model loss (categorical cross entropy) decreases but when I'm measuring <...
3
votes
0answers
34 views

A Deep CNN model delivering better results with standardization, when compared with normalization

I developed a deep CNN model, based on the architecture discussed in this paper, to generate predictions for time series data. My training data is shown in the figure below: In order to train the ...
3
votes
0answers
173 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
2answers
2k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
3
votes
0answers
262 views

How to reload cell in jupyter notebook? After OSError: [Errno 12] Cannot allocate memory

I got an error: OSError: [Errno 12] Cannot allocate memory I deleted some files. And I have free memory. I don't want start learning again from first epoch. (I got ...
3
votes
0answers
579 views

Deep Reinforcement Learning for dynamic pricing

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the ...
3
votes
0answers
215 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
1answer
602 views

Different learning rate for each of the layers?

I noticed that some popular deep learning frameworks like Keras or Pytorch allow you to set different learning rate for each layer. What are the benefits of that approach?
3
votes
0answers
155 views

Training deep CNN with noisy dataset

I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering. The train set captures ...
3
votes
1answer
244 views

Dataset image size and inference speed

Does training/fine tuning a pre-trained model on a the same dataset but with sizes scaled down (e.g., by 70%) improve inference speed? More generally, does training a CNN on smaller images improve ...
3
votes
1answer
362 views

Comparison between addition and multiplication function in deep neural network?

I designed a specific Convolution Neural Network to study in the area of image processing. The network has a part that there are two tensors which have to be transformed into a tensor in order to be ...
3
votes
1answer
126 views

Can a neural network recognize a letter B as an A if your trained it so?

You have a neural network. And you have, say, pictures of $100,000$ hand-written letters (A-Z). Now you make a typical Training and the neural network will recognize an A as an A, a B as a B, ... Now ...
3
votes
1answer
2k views

ValueError: Numpy arrays that you are passing to your model is not the size the model expected

I am trying to perform concatenation on the Bidirectinal LSTM layer. I have my model defined like this: ...
3
votes
0answers
115 views

How many pairs of image needed for training Siamese Network? And how to augment them?

Hi Deep Learning researchers and engineers, Does anyone have experience in Siamese Network regarding the training data size? How many pairs of image do I need to train a Siamese Network? And what ...
3
votes
0answers
873 views

Cross validation for convolutional neural network

I am using Keras to create a CNN model, and I would to use K-fold cross-validation to train the dataset. The dataset contains images and I am using ...
3
votes
0answers
273 views

convolutional neural network with cross validation in Keras

I want to use K-fold cross-validation on my dataset of images. I am reading the data (images) from a directory. How do I use cross validation with convolutional neural network in Keras?
3
votes
0answers
931 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
3
votes
1answer
213 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...
3
votes
0answers
115 views

Why does joint embedding of word and images work?

I often see some papers where the authors do point-wise multiplication of word and image embedding (e.g the image below). Why does this implementation works? I do not understand.
3
votes
1answer
619 views

Does Pooling remove spatial information of image in CNN?

Pr. Geoffrey Hinton has pointed out that pooling-layers remove spatial feature information. But, essentially, does the process that last convolutional layer's features are flattened for FC layer makes ...

1
2 3 4 5
26