Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,294 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
11
votes
0answers
1k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
6
votes
2answers
103 views

how to generate automatically images meshing up different shapes with a deep learning software?

My pursuit is to generate something like a grottesque(a kind of painting producing human-animals and plants hybrids). I need to do something like this painting in order to create an art exhibition. I ...
5
votes
1answer
153 views

Can a neural network recognize a letter B as an A if your trained it so?

You have a neural network. And you have, say, pictures of $100,000$ hand-written letters (A-Z). Now you make a typical Training and the neural network will recognize an A as an A, a B as a B, ... Now ...
5
votes
0answers
1k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
5
votes
1answer
1k views

How to draw a simple LSTM network

I'm new to deep learning, I am learning LSTM for my PhD work. This is a simple LSTM network for sequence classification. This code is from MATLAB tutorial: ...
5
votes
1answer
133 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
4
votes
2answers
104 views

How propagate the error delta in backpropagation in convolutional neural networks (CNN)?

My CNN has the following structure: Output neurons: 10 Input matrix (I): 28x28 Convolutional layer (C): 3 feature maps with a 5x5 kernel (output dimension is 3x24x24) Max pooling layer (MP): size 2x2 ...
4
votes
0answers
165 views

How to detect vanishing and exploding gradients with Tensorboard?

I have two "sub-questions" 1) How can I detect vanishing or exploding gradients with Tensorboard, given the fact that currently write_grads=True is deprecated in the Tensorboard callback as per "un-...
4
votes
0answers
430 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My programme gives the following error message: ...
4
votes
0answers
113 views

What is the relationship between “landmark Detection” and “landmark localization”

I am reading this paper "Grand Challenge of 106-Point Facial Landmark Localization" In the context of face recognition "Landmark Detection" is to detect a face by matching landmarks on a face. "...
4
votes
2answers
3k views

GAN - am I seeing mode collapse? Common fixes not working

I have a 2 part question. Context I am learning about GANs and writing my own starting from the very simplest example of adversarial learning (1-parameter node), then implementing a very simple 1-...
4
votes
2answers
3k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
4
votes
1answer
325 views

How to reload cell in jupyter notebook? After OSError: [Errno 12] Cannot allocate memory

I got an error: OSError: [Errno 12] Cannot allocate memory I deleted some files. And I have free memory. I don't want start learning again from first epoch. (I got ...
4
votes
1answer
471 views

Comparison between addition and multiplication function in deep neural network?

I designed a specific Convolution Neural Network to study in the area of image processing. The network has a part that there are two tensors which have to be transformed into a tensor in order to be ...
4
votes
1answer
93 views

How to train neural word embeddings?

So I am new to Deep Learning and NLP. I have read several blog posts on medium, towardsdatascience and papers where they talk about pre-training the word embeddings in an unsupervised fashion and then ...
4
votes
1answer
899 views

Implementing spatio-temporal convolutions in pytorch

I am trying to implement a layer to perform the (2+1)D convolutions described in this paper: https://arxiv.org/pdf/1711.11248.pdf The basic idea is as follows: Let's say I have a 3D convolutional ...
4
votes
1answer
45 views

Is there any work done on reconfigurable convolutional neural networks?

Convolutional Neural networks are used in supervised learning meaning models are always "set in stone" after training (architecture and paramters) so this might not even be possible, but is there any ...
4
votes
0answers
276 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
151 views

How do we pass data to a RNN?

Let's say we have A1, A2, ... , Am different articles in the corpus and each of them has W1, W2, ....., Ww words. We are training a language model on them. Do we: Scheme 1 Take the first batch of ...
4
votes
1answer
458 views

Multi-task learning for Multi-label classification?

I have a multi-label classification problem wherein each example can belong to one of the pre-defined classes (or can belong to none of them). I was wondering if I can somehow apply multi-task ...
4
votes
1answer
2k views

Gradient flow through concatenation operation

I need help in understanding the gradient flow through a concatenation operation. I'm implementing a network (mostly a CNN) which has a concatenation operation (in pytorch). The network is defined ...
4
votes
0answers
263 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
1answer
138 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
3
votes
0answers
16 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
3
votes
0answers
32 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
3
votes
0answers
38 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
3
votes
0answers
132 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
3
votes
1answer
348 views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
3
votes
1answer
37 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
3
votes
2answers
39 views

User profiling based on multiple posts

I currently have collected a dataset of different social media posts for each user with labels assigned to each user. I tried to use LSTM, and BERT for the text classification problem, So for each ...
3
votes
1answer
2k views

SymbolicException: Inputs to eager execution function cannot be Keras symbolic tensors

I am writing Encoder-Decoder architecture with Bahdanau Attention using tf.keras with TensorFlow 2.0. Below is my code This is working with TensorFlow 1.15 but getting the error in 2.0. you can check ...
3
votes
0answers
32 views

Keras model with second to last sigmoid activated Conv1D layer followed by globalMaxPool outputs values outside [0,1]. Why?

I am trying to train a binary classifier. It is a residual network with skip layers etc. but ultimately, the bottom two layers are a 1D convolution with sigmoid activation followed by a global max ...
3
votes
1answer
636 views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
3
votes
0answers
34 views

Why is a general/original softmax loss not preferred in FR (face recognition)?

In some papers I've read that softmax loss is not preferred in FR since it does not give a good inter-class and intra-class margins, but could not understand 'why?'. So can someone explain, why ...
3
votes
1answer
442 views

How to use PCA in CNN for image recognition using Keras?

I created a CNN model for image classification and I want to use Principal Component Analysis (PCA) but when I run pca.fit() code, the code still running for hours ...
3
votes
0answers
114 views

Why categorical cross entropy loss is not correlated with NLP scores?

I'm training a deep network for image captioning which is consist of one CNN and three GRUs. During training epoch by epoch model loss (categorical cross entropy) decreases but when I'm measuring <...
3
votes
0answers
34 views

A Deep CNN model delivering better results with standardization, when compared with normalization

I developed a deep CNN model, based on the architecture discussed in this paper, to generate predictions for time series data. My training data is shown in the figure below: In order to train the ...
3
votes
0answers
50 views

Train ML algorithm to find edges

I have input RGB images as follows: I have a dataset of manually annotated images highlighting the outline(edges) from the input images I am attaching an example. My aim is to train a ML algorithm ...
3
votes
0answers
192 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
3
votes
0answers
911 views

One hot encoding as input to recurrent neural networks

I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels Example of input pattern: ...
3
votes
0answers
112 views

How to design the generator in generative adversarial network (GAN)?

I am new to GAN. And recently read a paper about how to implement GAN in recommender system here I have a question in the paper. The Eq (20) in the paper should ...
3
votes
0answers
622 views

Deep Reinforcement Learning for dynamic pricing

I am trying to implement a Deep Q Network model for Dynamic pricing in Logistics. I can define State Space (Origin, Destination, type of the shipment, customer, Type of the product, Commodity of the ...
3
votes
0answers
220 views

convLSTM : how to structure input data

I have the following dataframe containing training data that I have been using to perform a regression task using CNN + FC : ...
3
votes
0answers
230 views

Kronecker product using sparse matrices / tensors in tensorflow

I am currently trying to perform a kronecker product on a pair of sparse tensors in tensorflow. I have found some code on another thread to perform the kronecker product using normal tensors. ...
3
votes
0answers
162 views

Training deep CNN with noisy dataset

I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering. The train set captures ...
3
votes
2answers
334 views

Where does the “deep learning needs big data” rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
3
votes
1answer
286 views

Dataset image size and inference speed

Does training/fine tuning a pre-trained model on a the same dataset but with sizes scaled down (e.g., by 70%) improve inference speed? More generally, does training a CNN on smaller images improve ...
3
votes
1answer
222 views

LeNet-5 - combining feature maps in C3 layer

Famous LeNet-5 architecture looks like this: The output of layer S2 has dimension: 10x10x6 - so basically an image with 6 convultions applied to it to derive features. If each dimension was again ...
3
votes
2answers
1k views

Gradient of NN output with respect to inputs

I've trained a neural network (NN) on a problem where multiple inputs can be mapped to the same output. I'd like to use this NN to go from an output to an input i.e. given an output vector $y$, I want ...
3
votes
1answer
236 views

How to train millions of doc2vec embeddings using GPU?

I am trying to train a doc2vec based on user browsing history (urls tagged to user_id). I use chainer deep learning framework. There are more than 20 millions (user_id and urls) of embeddings to ...

1
2 3 4 5
26