Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

1,571 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
17
votes
0answers
2k views

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory ...
7
votes
0answers
2k views

Why is my Keras model not learning image segmentation?

Edit: as is turns out, not even the model's initial creator could successfully fine-tune it. This is most likely a problem of implementation, or possibly related to the non-intuitive way in which the ...
6
votes
1answer
137 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
2answers
85 views

Imbalanced training set vs smaller balanced training set?

Say I am using a maximum likelihood approach and my output unit computes a softmax function. My training set is distributed as follows over 6 classes: ...
5
votes
0answers
360 views

How to detect vanishing and exploding gradients with Tensorboard?

I have two "sub-questions" 1) How can I detect vanishing or exploding gradients with Tensorboard, given the fact that currently write_grads=True is deprecated in the Tensorboard callback as per "un-...
5
votes
2answers
69 views

Reasoning behind using Deep Learning on non-local data

I understand the using of deep learning for data that have "local" structure, for example, images/videos/texts, as the convolutional layers reduce the amount of dimensions. However, I saw ...
5
votes
0answers
1k views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
5
votes
0answers
137 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
5
votes
1answer
3k views

Gradient flow through concatenation operation

I need help in understanding the gradient flow through a concatenation operation. I'm implementing a network (mostly a CNN) which has a concatenation operation (in pytorch). The network is defined ...
5
votes
0answers
1k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
4
votes
1answer
409 views

Self-attention mechanism did not improve the LSTM classification model

I am doing an 8-class classification using time series data. It appears that the implementation of the self-attention mechanism has no effect on the model so I think my implementations have some ...
4
votes
2answers
421 views

How propagate the error delta in backpropagation in convolutional neural networks (CNN)?

My CNN has the following structure: Output neurons: 10 Input matrix (I): 28x28 Convolutional layer (C): 3 feature maps with a 5x5 kernel (output dimension is 3x24x24) Max pooling layer (MP): size 2x2 ...
4
votes
0answers
360 views

How to use multiple text features for NLP classifier?

I am trying to build text classifier, Usually, we have one text column and ground truth. But I am working on a problem where dataset contains many text features. I am exploring different ways how to ...
4
votes
1answer
83 views

Is it possible to solve Rubik's cube using DQN?

I'm trying to solve Rubik's cube using deep learning and I came across with DQN, so I decided to give it a try. I developed all the code and started training but I got this results: Loss goes up and ...
4
votes
2answers
551 views

How does ResNet bottleneck architecture's input size is possible to change from 56x56x64 to 56x56x356?

In ResNet papaer, First residual block's input size is 56x56x64 caused by 7x7x64 filter in first layer. But, in the paper, they showed residual block that has 56x56x256 input size. How does it is ...
4
votes
1answer
1k views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
4
votes
0answers
765 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My program gives the following error message: ...
4
votes
0answers
155 views

Why do BERT classification do worse with longer sequence length?

I've been experimenting using transformer networks like BERT for some simple classification tasks. My tasks are binary assignment, the datasets are relatively balanced, and the corpus are abstracts ...
4
votes
1answer
825 views

How to use PCA in CNN for image recognition using Keras?

I created a CNN model for image classification and I want to use Principal Component Analysis (PCA) but when I run pca.fit() code, the code still running for hours ...
4
votes
2answers
5k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
4
votes
1answer
496 views

Calculation of Output in LSTM Many-to-One Architecture

I'm new to Recurrent Neural Network but I want to train my data with LSTM but I'm having a trouble to understand LSTM Many-to-One architecture. Suppose the size of my data is ...
4
votes
2answers
92 views

What are the reasons for drawing initial neural network weights from the Gaussian distribution?

Are there theoretical or empirical reasons for drawing initial weights of a multilayer perceptron from a Gaussian rather than from, say, a Cauchy distribution?
4
votes
2answers
205 views

Benefits of using Deep Learning-specific hyperparameter optimization tools vs. sklearn?

There are quite a few library for hyperparameter optimization that are specific to Keras or other Deep Learning libraries, like Hyperas or Talos. My question is, what's the main benefit of using ...
4
votes
0answers
1k views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
4
votes
0answers
304 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
180 views

How do we pass data to a RNN?

Let's say we have A1, A2, ... , Am different articles in the corpus and each of them has W1, W2, ....., Ww words. We are training a language model on them. Do we: Scheme 1 Take the first batch of ...
4
votes
0answers
270 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
1answer
199 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
4
votes
2answers
657 views

How to determine the number of the training images in Keras after data augmentaion?

I want to create a CNN model and I am using data augmentation. I want know the number of augmented images in Keras. How to determine the number of the training images in Keras after data augmentation?...
3
votes
0answers
24 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
3
votes
0answers
37 views

Why margin loss is used in Capsule Network instead of Cross Entropy loss?

I'm reading the Capsule Network paper proposed by Hinton. I'm not sure why the margin loss is used instead of the cross entropy loss. Any intuitive explaination for this?
3
votes
1answer
39 views

Keras use of ImageNet?

Keras mentions that it provided models pretrained on ImageNet. However, it doesn't specify what they mean by "ImageNet" - like is it a certain subset of ImageNet of the complete set of ...
3
votes
0answers
55 views

Using softmax for multilabel classification (as per Facebook paper)

I came across this paper by some Facebook researchers where they found that using a softmax and CE loss function during training led to improved results over sigmoid + BCE. They do this by changing ...
3
votes
2answers
503 views

Why is the "dying ReLU" problem not present in most modern deep learning architectures?

The $ReLU(x) = max(0,x)$ function is an often used activation function in neural networks. However it has been shown that it can suffer from the dying Relu problem (see also What is the "dying ...
3
votes
1answer
97 views

Deep learning test loss curve won't go down

I've been working with Deep Learning projects for this current project that I am working on and it's basically a time series classification problem. Where given an array of time series data I need to ...
3
votes
0answers
178 views

What are the 'protos' in TF Object Detection?

I am struggling to understand what are the 'protos' in TF Object Detection? Why do we need them here? Also, while setting up the TF API we need to download and compile protocol buffers. There is also ...
3
votes
1answer
66 views

How to train YOLOV4?

I am going to write yolov4 real-time object detection, and I have to do it for car then vehicle plate number, but it does not have to find plate number if there is no car, first car then number on car,...
3
votes
2answers
189 views

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) ...
3
votes
1answer
93 views

How to specify version for dependencies so that each one is compatible and stays within a size limit?

I am trying to deploy a web app to Heroku. The free tier is limited to 500 MB. I am using my resnet34 model as a .pkl file. I create model with it using the fastai ...
3
votes
0answers
18 views

How respective gating functions are ensured in LSTM?

I'm studying the Hochreiter-Schmidhuber long-short term memory recurrent architecture. The overall idea, information flow and manipulation is clear, and it seemingly works, but what I cannot ...
3
votes
0answers
88 views

AlexNet Research Paper VS PytTorch and Tensorflow implementation

I'm making my way through Deep Learning research papers, starting with AlexNet, and I found differences in the implementation of PyTorch and Tensorflow that I can't explain. In the research paper, ...
3
votes
0answers
41 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
3
votes
1answer
205 views

Custom connections between layers Keras

I would like to manually define connections in neural network between layers using TensorFlow and keras with Python. By default connections are beween all pairs of neurons. I need to make connections ...
3
votes
0answers
234 views

Chess deep learning siamese network overfitting when shouldn't in theory

TLDR: My network is training with pairs so instead of 10^6 samples it has 10^12 samples (The number of samples squared) . With that large of a data set is shouldn't overfit but it does after very few ...
3
votes
2answers
253 views

How to perform node classification using Graph Neural Networks

I'm am trying to perform node classification using graph neural network methods. My initial plan was to convert my graphs to adjacency matrices and train my network on that, with the node features ...
3
votes
1answer
52 views

Facial recognition architecture

Image recognition uses deep learning, and in particular CNNs to train on and recognise faces. Usually, this entails training on lots of data. However, recently, we have seen face recognition being ...
3
votes
2answers
96 views

How can we create an label, value detector?

I am trying to implement an text detector using MaskRCNN such that the model detects the label and value as shown in the image below. Detecting the same is easier for fields like page date and order ...
3
votes
1answer
2k views

Can not install spacy package on windows 10 via pip

I have below environment. OS: Windows 10 Python: Python 3.7.4 PIP: pip 19.3.1 I am trying to install spacy in my windows 10 OS. It gives me below error. ...
3
votes
1answer
401 views

Discriminator of a Conditional GAN with continuous labels

OK, let's say we have well-labeled images with non-discrete labels such as brightness or size or something and we want to generate images based on it. If it were done with a discrete label it could ...
3
votes
1answer
97 views

Training a sound localization neural network

I am trying to train a neural network, to estimate the location (in degrees from 0 to 180) a sound is coming from. I am using TensorFlow Keras in python to train the model. The input data are two ...

1
2 3 4 5
32