Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,662 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
22 votes
1 answer
3k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
user avatar
9 votes
0 answers
2k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
Matt's user avatar
  • 199
8 votes
1 answer
300 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
lhk's user avatar
  • 181
7 votes
0 answers
1k views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
Aurast's user avatar
  • 171
7 votes
0 answers
2k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
Varun's user avatar
  • 71
6 votes
1 answer
2k views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
user1253952's user avatar
6 votes
0 answers
378 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
Jason's user avatar
  • 61
6 votes
0 answers
291 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
Shindou's user avatar
  • 161
5 votes
1 answer
1k views

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
Adam's user avatar
  • 896
5 votes
0 answers
1k views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My program gives the following error message: ...
Kehrwert's user avatar
  • 163
5 votes
1 answer
435 views

Time horizon T in policy gradients (actor-critic)

I am currently going through the Berkeley lectures on Reinforcement Learning. Specifically, I am at slide 5 of this lecture. At the bottom of that slide, the gradient of the expected sum of rewards ...
Dummie Variable's user avatar
4 votes
0 answers
62 views

Why did I got opposite results of the original "How transferable are features in deep neural networks" paper?

I got tasked with reproducing the results of the influential "How transferable are features in deep neural networks?" paper in a DL class I'm taking (Full code). I got the exact opposite ...
OfirD's user avatar
  • 91
4 votes
1 answer
2k views

Self-attention mechanism did not improve the LSTM classification model

I am doing an 8-class classification using time series data. It appears that the implementation of the self-attention mechanism has no effect on the model so I think my implementations have some ...
Leo's user avatar
  • 125
4 votes
2 answers
1k views

How does ResNet bottleneck architecture's input size is possible to change from 56x56x64 to 56x56x356?

In ResNet papaer, First residual block's input size is 56x56x64 caused by 7x7x64 filter in first layer. But, in the paper, they showed residual block that has 56x56x256 input size. How does it is ...
douner's user avatar
  • 115
4 votes
0 answers
612 views

How to use a ragged tensor with a convolutional layer?

I have textual data of various lengths for which ragged tensors seems well suited. For instance my data could look as follows : ...
pierre_sendorek's user avatar
4 votes
2 answers
7k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
Ben Groene's user avatar
4 votes
0 answers
235 views

Training deep CNN with noisy dataset

I am training a Mask RCNN model with a train dataset that has been generated from some simple computer vision operations (color thresholding) and some morphological filtering. The train set captures ...
Gouda's user avatar
  • 171
4 votes
4 answers
713 views

Where does the "deep learning needs big data" rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
Aran's user avatar
  • 41
4 votes
1 answer
1k views

Calculating saliency maps for text classification

I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most. I ...
Marc Jones's user avatar
4 votes
2 answers
238 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
Edgar Derby's user avatar
4 votes
1 answer
310 views

How does Pooling Layer in CNN introduce invariance to other transformations besides translation

Here is a quote from deeplearningbook which I am trying to process. I am not sure what do they mean by this quote, can someone help me understand please? Pooling over spatial regions produces ...
Stefan Radonjic's user avatar
4 votes
3 answers
937 views

How to determine the number of the training images in Keras after data augmentaion?

I want to create a CNN model and I am using data augmentation. I want know the number of augmented images in Keras. How to determine the number of the training images in Keras after data augmentation?...
N.IT's user avatar
  • 1,995
3 votes
0 answers
139 views

Intuitively, why do Non-monotonic Activations Work?

The swish/SiLU activation is very popular, and many would argue it has dethroned ReLU. However, it is non-monotonic, which seems to go against popular intuition (at least on this site: example 1, ...
Jason's user avatar
  • 53
3 votes
1 answer
126 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
Alpha's user avatar
  • 31
3 votes
0 answers
240 views

Why margin loss is used in Capsule Network instead of Cross Entropy loss?

I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
xtiger's user avatar
  • 131
3 votes
0 answers
249 views

when depthwise separable convolution should be preferred over normal convolution?

As a novice in the realm of deep learning, I recently learned about Depthwise Separable Convolution. I have seen some tutorials and articles about it on internet, and in all of them the author ...
K327's user avatar
  • 131
3 votes
1 answer
308 views

VQ-GAN understanding

I tried to understand how VQ-GAN works, but unfortunately I have not understood it. I tried to read some articles about it and watch a video. I believe a good and simple article will help me. You ...
alex-uarent-alex's user avatar
3 votes
1 answer
484 views

What are the 'protos' in TF Object Detection?

I am struggling to understand what are the 'protos' in TF Object Detection? Why do we need them here? Also, while setting up the TF API we need to download and compile protocol buffers. There is also ...
user109348's user avatar
3 votes
1 answer
139 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
truth's user avatar
  • 280
3 votes
0 answers
124 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
Begoodpy's user avatar
  • 223
3 votes
1 answer
74 views

What's the best way to validate a rare event detection model during training?

When training a deep model for rare event detection (e.g. sound of an alarm in a home device audio stream), is it best to use a balanced validation set (50% alarm, 50% normal) to determine early ...
jack's user avatar
  • 61
3 votes
0 answers
53 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
mkanakis's user avatar
  • 131
3 votes
1 answer
240 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
Javier Jiménez de la Jara's user avatar
3 votes
0 answers
782 views

Understanding depthwise convolution vs convolution with group parameters in pytorch

So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows. For a input feature map of (C_in, F_in, F_in), we take only 1 ...
lincr's user avatar
  • 91
3 votes
0 answers
357 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
EXTORY's user avatar
  • 31
3 votes
1 answer
243 views

stacking features vs concatenating layers

I am trying to get to the logical intuition of differences between stacking multiple features and passing it via a final block (which could comprise multiple layers and lets say a final classification ...
Vikram Murthy's user avatar
3 votes
3 answers
501 views

How to perform node classification using Graph Neural Networks

I'm am trying to perform node classification using graph neural network methods. My initial plan was to convert my graphs to adjacency matrices and train my network on that, with the node features ...
Andrew's user avatar
  • 179
3 votes
1 answer
50 views

Approach to classify blocks of time series

I am wondering if there exists an approach to classify blocks of time series, and not specifically individual time series. If so, can you point me out papers/articles/tutorials where these type of ...
YellowishLight's user avatar
3 votes
1 answer
82 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
user's user avatar
  • 2,003
3 votes
2 answers
157 views

How can we create an label, value detector?

I am trying to implement an text detector using MaskRCNN such that the model detects the label and value as shown in the image below. Detecting the same is easier for fields like page date and order ...
hR 312's user avatar
  • 91
3 votes
1 answer
767 views

Discriminator of a Conditional GAN with continuous labels

OK, let's say we have well-labeled images with non-discrete labels such as brightness or size or something and we want to generate images based on it. If it were done with a discrete label it could ...
user3023715's user avatar
3 votes
0 answers
109 views

Keras model with second to last sigmoid activated Conv1D layer followed by globalMaxPool outputs values outside [0,1]. Why?

I am trying to train a binary classifier. It is a residual network with skip layers etc. but ultimately, the bottom two layers are a 1D convolution with sigmoid activation followed by a global max ...
user3075342's user avatar
3 votes
2 answers
138 views

ConvNet with concatenated data

I have a basic question regarding convolutional neural network. Assume I have a set of 1000 RGB images and I train a CNN from this set. I can obviously split each of my RGB images into 3 different ...
ev5071's user avatar
  • 31
3 votes
0 answers
44 views

Deep learning and label noise. Best practices for the real world

Unlike MNIST or other benchmark datasets collected data often come with subpar, inaccurate labels. What are the best practices to help the neural networks to don't overfit the noise? Things that comes ...
ajeje's user avatar
  • 191
3 votes
1 answer
98 views

What Models should i try for this problem?

I need some advice for a problem i'm working on with automobile data. The vehicles provide a series of codes at every second which are bieng stored, though it can vary how many. For example , at time ...
Parth Sindhu's user avatar
3 votes
0 answers
48 views

A Deep CNN model delivering better results with standardization, when compared with normalization

I developed a deep CNN model, based on the architecture discussed in this paper, to generate predictions for time series data. My training data is shown in the figure below: In order to train the ...
chupa_kabra's user avatar
3 votes
1 answer
699 views

Why are parameter updates downscaled by uncentered variance (instead of centered variance) in Adam optimizer?

In Adam optimizer algorithm, parameter updates are computed as follows: $\theta_t \leftarrow \theta_{t-1} - \alpha \frac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}$ Where $\hat{m}_t$ is a bias-corrected ...
Charles Lagace's user avatar
3 votes
0 answers
322 views

How do I implement masking in TensorFlow eager execution?

I am training a stateful RNN on variable length sequences (optional: see my previous question for more details). I padded the sequences to a fixed length with the value -1. The when batches are ...
DankMasterDan's user avatar
3 votes
1 answer
3k views

One hot encoding as input to recurrent neural networks

I'm trying to predict next label in a pattern based on previous labels using recurrent neural network. In total I have 100 labels Example of input pattern: ...
Ammar Ahmed's user avatar
3 votes
0 answers
153 views

How to design the generator in generative adversarial network (GAN)?

I am new to GAN. And recently read a paper about how to implement GAN in recommender system here I have a question in the paper. The Eq (20) in the paper should ...
jason's user avatar
  • 329

1
2 3 4 5
34