Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,762 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
21 votes
0 answers
3k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
user avatar
8 votes
0 answers
2k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
user avatar
  • 189
7 votes
0 answers
165 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
user avatar
  • 171
6 votes
0 answers
1k views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
user avatar
  • 161
6 votes
0 answers
1k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
user avatar
  • 61
5 votes
1 answer
717 views

How to use multiple text features for NLP classifier?

I am trying to build text classifier, Usually, we have one text column and ground truth. But I am working on a problem where dataset contains many text features. I am exploring different ways how to ...
user avatar
5 votes
3 answers
91 views

Reasoning behind using Deep Learning on non-local data

I understand the using of deep learning for data that have "local" structure, for example, images/videos/texts, as the convolutional layers reduce the amount of dimensions. However, I saw ...
user avatar
  • 291
5 votes
2 answers
125 views

What are the reasons for drawing initial neural network weights from the Gaussian distribution?

Are there theoretical or empirical reasons for drawing initial weights of a multilayer perceptron from a Gaussian rather than from, say, a Cauchy distribution?
user avatar
5 votes
0 answers
2k views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
user avatar
5 votes
0 answers
338 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
user avatar
  • 51
5 votes
1 answer
3k views

Gradient flow through concatenation operation

I need help in understanding the gradient flow through a concatenation operation. I'm implementing a network (mostly a CNN) which has a concatenation operation (in pytorch). The network is defined ...
user avatar
  • 151
5 votes
0 answers
280 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
user avatar
  • 151
4 votes
1 answer
843 views

Self-attention mechanism did not improve the LSTM classification model

I am doing an 8-class classification using time series data. It appears that the implementation of the self-attention mechanism has no effect on the model so I think my implementations have some ...
user avatar
  • 115
4 votes
3 answers
2k views

Bert-Transformer : Why Bert transformer uses [CLS] token for classification instead of average over all tokens?

I am doing experiments on bert architecture and found out that most of the fine-tuning task takes the final hidden layer as text representation and later they pass it to other models for the further ...
user avatar
4 votes
2 answers
635 views

How propagate the error delta in backpropagation in convolutional neural networks (CNN)?

My CNN has the following structure: Output neurons: 10 Input matrix (I): 28x28 Convolutional layer (C): 3 feature maps with a 5x5 kernel (output dimension is 3x24x24) Max pooling layer (MP): size 2x2 ...
user avatar
  • 99
4 votes
2 answers
696 views

How does ResNet bottleneck architecture's input size is possible to change from 56x56x64 to 56x56x356?

In ResNet papaer, First residual block's input size is 56x56x64 caused by 7x7x64 filter in first layer. But, in the paper, they showed residual block that has 56x56x256 input size. How does it is ...
user avatar
  • 115
4 votes
1 answer
2k views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
user avatar
  • 476
4 votes
0 answers
879 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My program gives the following error message: ...
user avatar
  • 153
4 votes
2 answers
6k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
user avatar
4 votes
3 answers
533 views

Where does the "deep learning needs big data" rule come from

When reading about deep learning I often come across the rule that deep learning is only effective when you have large amounts of data at your disposal. These statements are generally accompanied by a ...
user avatar
  • 41
4 votes
1 answer
961 views

Calculating saliency maps for text classification

I'm following the text classification with movie reviews TensorFlow tutorial, and wanted to extend the project by looking, for a certain input, which words influenced the classification the most. I ...
user avatar
4 votes
2 answers
218 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
user avatar
4 votes
1 answer
225 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
user avatar
  • 1,237
4 votes
1 answer
265 views

How does Pooling Layer in CNN introduce invariance to other transformations besides translation

Here is a quote from deeplearningbook which I am trying to process. I am not sure what do they mean by this quote, can someone help me understand please? Pooling over spatial regions produces ...
user avatar
4 votes
2 answers
719 views

How to determine the number of the training images in Keras after data augmentaion?

I want to create a CNN model and I am using data augmentation. I want know the number of augmented images in Keras. How to determine the number of the training images in Keras after data augmentation?...
user avatar
  • 1,765
3 votes
1 answer
42 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
user avatar
  • 31
3 votes
0 answers
79 views

Why margin loss is used in Capsule Network instead of Cross Entropy loss?

I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
user avatar
  • 131
3 votes
1 answer
144 views

Using softmax for multilabel classification (as per Facebook paper)

I came across this paper by some Facebook researchers where they found that using a softmax and CE loss function during training led to improved results over sigmoid + BCE. They do this by changing ...
user avatar
3 votes
1 answer
289 views

Explanation of why Neural Networks are non convex

Why having a symmetry of values for the hidden state imply that the neural network is non convex? I could not find an intuitive answer for this yet. Also, if we consider a Fully Connected network wtih ...
user avatar
  • 141
3 votes
0 answers
267 views

What are the 'protos' in TF Object Detection?

I am struggling to understand what are the 'protos' in TF Object Detection? Why do we need them here? Also, while setting up the TF API we need to download and compile protocol buffers. There is also ...
user avatar
3 votes
1 answer
98 views

How to train YOLOV4?

I am going to write yolov4 real-time object detection, and I have to do it for car then vehicle plate number, but it does not have to find plate number if there is no car, first car then number on car,...
user avatar
3 votes
3 answers
257 views

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) ...
user avatar
  • 31
3 votes
0 answers
212 views

Multiple features in LSTM

It's clear how LSTM works with 1 feature. But what happens if the number of features is > 1? According to the answer proposed here, Keras creates a computational graph that executes the sequence ...
user avatar
  • 157
3 votes
1 answer
110 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
user avatar
  • 180
3 votes
0 answers
100 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
user avatar
  • 223
3 votes
0 answers
42 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
user avatar
  • 131
3 votes
1 answer
114 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
user avatar
3 votes
0 answers
495 views

Understanding depthwise convolution vs convolution with group parameters in pytorch

So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows. For a input feature map of (C_in, F_in, F_in), we take only 1 ...
user avatar
  • 91
3 votes
0 answers
272 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
user avatar
  • 31
3 votes
2 answers
290 views

How to perform node classification using Graph Neural Networks

I'm am trying to perform node classification using graph neural network methods. My initial plan was to convert my graphs to adjacency matrices and train my network on that, with the node features ...
user avatar
  • 169
3 votes
1 answer
35 views

Approach to classify blocks of time series

I am wondering if there exists an approach to classify blocks of time series, and not specifically individual time series. If so, can you point me out papers/articles/tutorials where these type of ...
user avatar
3 votes
1 answer
56 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
user avatar
  • 1,881
3 votes
2 answers
107 views

How can we create an label, value detector?

I am trying to implement an text detector using MaskRCNN such that the model detects the label and value as shown in the image below. Detecting the same is easier for fields like page date and order ...
user avatar
  • 81
3 votes
1 answer
521 views

Discriminator of a Conditional GAN with continuous labels

OK, let's say we have well-labeled images with non-discrete labels such as brightness or size or something and we want to generate images based on it. If it were done with a discrete label it could ...
user avatar
3 votes
0 answers
78 views

Keras model with second to last sigmoid activated Conv1D layer followed by globalMaxPool outputs values outside [0,1]. Why?

I am trying to train a binary classifier. It is a residual network with skip layers etc. but ultimately, the bottom two layers are a 1D convolution with sigmoid activation followed by a global max ...
user avatar
3 votes
0 answers
421 views

How to use a ragged tensor with a convolutional layer?

I have textual data of various lengths for which ragged tensors seems well suited. For instance my data could look as follows : ...
user avatar
3 votes
2 answers
86 views

ConvNet with concatenated data

I have a basic question regarding convolutional neural network. Assume I have a set of 1000 RGB images and I train a CNN from this set. I can obviously split each of my RGB images into 3 different ...
user avatar
  • 31
3 votes
0 answers
39 views

Deep learning and label noise. Best practices for the real world

Unlike MNIST or other benchmark datasets collected data often come with subpar, inaccurate labels. What are the best practices to help the neural networks to don't overfit the noise? Things that comes ...
user avatar
  • 191
3 votes
1 answer
72 views

What Models should i try for this problem?

I need some advice for a problem i'm working on with automobile data. The vehicles provide a series of codes at every second which are bieng stored, though it can vary how many. For example , at time ...
user avatar
3 votes
0 answers
29 views

Best practices for scaling data science / engineering teams

I am trying to find best practices for scaling data science teams, i.e find an efficient workflow/methodology to divide work between Software Engineers and Researchers working on a same product. I’...
user avatar
  • 131

1
2 3 4 5
36