Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
2
votes
1answer
329 views

Deep learning facial recognition research project

I'm a second year pure maths, applied maths and computer science student. I have taken up a research course and was given the topic to focus on facial recognition using deep learning. I have done a ...
1
vote
0answers
111 views

What did DeepMind do with AlphaGo between the Fan Hui and Lee Sedol games?

In January, DeepMind published the article (see video) about its win against Fan Hui, which happend in October 2015. The article and other interviews say, it used 100.000 human games, and then 13.000....
1
vote
3answers
457 views

Training multi-label classifier with low quality training set

So I'm creating a topics classifier where a document may be tagged for several different topics, let's say - A, B while actually the document belongs to A, B and C. In the training stage I want the ...
53
votes
2answers
49k views

When to use (He or Glorot) normal initialization over uniform init? And what are its effects with Batch Normalization?

I knew that Residual Network (ResNet) made He normal initialization popular. In ResNet, He normal initialization is used , while the first layer uses He uniform initialization. I've looked through ...
1
vote
0answers
48 views

Find boundaries for a smallest mean error

I am new at deep learning, but willing to learn. I have this problem. Have 5 inputs: A,B,C,D,E(columns in CSV) that gives me 1 or 0. A,B,C,D or E can be between 0 and 100. For example, a ...
3
votes
2answers
3k views

How Int8 (byte) operations can be useful for deep learning?

Nvidia is planing to add hardware support for int8 operations to their titan card and target deep learning. I am trying to understood how its useful and what types of network will benefit from this. ...
31
votes
1answer
10k views

Paper: What's the difference between Layer Normalization, Recurrent Batch Normalization (2016), and Batch Normalized RNN (2015)?

So, recently there's a Layer Normalization paper. There's also an implementation of it on Keras. But I remember there are papers titled Recurrent Batch Normalization (Cooijmans, 2016) and Batch ...
4
votes
0answers
247 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
87
votes
15answers
69k views

How do you visualize neural network architectures?

When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture. What are good / simple ways to visualize common ...
1
vote
1answer
120 views

which application domains are not well suited to deep learning?

Deep learning seems to be the new cool thing in AI/machine learning and it works well in many domains, but I want to know- what are the specific application areas where deep learning is not the best ...
0
votes
1answer
198 views

Deep learning - rule generation

I wanted to know if there is any methodology in Deep/Machine learning, where given a set of input/output values, it can derive rules for the same. Lets say I generate training input and output by $y=...
2
votes
1answer
152 views

LSTMs: what is $W_x$ & $U_z$ in $φ(W_x + U_z + b)$?

Reading On Multiplicative Integration with Recurrent Neural Networks Despite of their varying characteristics, most of them(RNNs) share a common computational building block, described by the ...
5
votes
2answers
6k views

Using RNN (LSTM) for predicting one future value of a time series

I have been reading several papers, articles and blog posts about RNNs (LSTM specifically) and how we can use them to do time series prediction. In almost all examples and codes I have found, the ...
66
votes
5answers
38k views

Time series prediction using ARIMA vs LSTM

The problem that I am dealing with is predicting time series values. I am looking at one time series at a time and based on for example 15% of the input data, I would like to predict its future values....
1
vote
2answers
2k views

Convert filters pre-trained with ImageNet to grayscale?

I am wondering how to convert caffe reference model trained with ImageNet (color pics) for grayscale image to save memory and to speed up. The filters in the caffe convolution layers are different ...
2
votes
2answers
3k views

Why does sigmoid/tanh activation function is still used for deep NN when we have ReLU?

Looks like ReLU is better then sigmoid or tanh for deep NN from all aspects: simple more biologically plausible no gradient to vanish better performance sparsity And I see only one advantage of ...
6
votes
2answers
1k views

Is there a known convolutional net architecture to calculate object masks for images?

I would like to train a convnet to do the following: Input is a set of single channel (from black to tones of grey to white) pictures with a given object, let's say cars. Target is, for every picture ...
17
votes
3answers
10k views

How to calculate the mini-batch memory impact when training deep learning models?

I'm trying to calculate the amount of memory needed by a GPU to train my model based on this notes from Andrej Karphaty: http://cs231n.github.io/convolutional-networks/#computational-considerations ...
1
vote
1answer
1k views

Theano vs Tensorflow for building Neural Networks for NLP tasks

I am trying to learn Theano and TensorFlow for building neural networks for NLP based tasks. Any suggestions as to when one should choose one over the other or what works better and when or is it just ...
4
votes
1answer
935 views

Recommendations and Missing Data in Deep Learning

In this research paper, it is discussed how to combine deep learning with wide (shallow) learning to achieve both generalisation and the ability to learn correlation/association rules. The input ...
40
votes
3answers
37k views

Does batch_size in Keras have any effects in results' quality?

I am about to train a big LSTM network with 2-3 million articles and am struggling with Memory Errors (I use AWS EC2 g2x2large). I found out that one solution is to reduce the ...
2
votes
1answer
58 views

Two ways of optimize the same function?

I'm actually reading this tutorial about deepLearning and in particular about Logistic Regression. I don't get why it first says to optimize logistic regression taking the max Probability and after ...
9
votes
1answer
3k views

How to approach the numer.ai competition with anonymous scaled numerical predictors?

Numer.ai has been around for a while now and there seem to be only few posts or other discussions about it on the web. The system has changed from time to time and the set-up today is the following: ...
3
votes
2answers
750 views

How to represent target variable for chess AI

Inspired by Google's recent AlphaGo project, I've decided that as a fun personal challenge I'd like to use deep learning and convoluted neural networks to build an algorithm that can beat an ordinary ...
1
vote
1answer
498 views

How to form Hessian matrix in BFGS Quasi-Newton Method

I came across this link. In BFGS Quasi-Newton Method, a Hessian matrix is used in weight updation. Is there any resource where I can find how this hessian matrix was obtained along with a clear ...
4
votes
1answer
1k views

Approaches for implementing Domain specific Question answering System

Given several wikipedia articles on different movies. What are the different approaches to implement a QA system to answer different quires related to movies. ...
3
votes
1answer
597 views

Deep Learning for Time series

Deep Learning is an excellent model for classification problem such as image recognition or object detection. Can we use deep learning for regression problems - Time Series prediction ? So if it can, ...
8
votes
1answer
1k views

Using Neural Networks to extract multiple parameters from images

I want to extract parameters from an image using a neural network. Example: Given an image of a brick wall the NN should extract the width and height of the bricks, the color and the roughness. I ...
5
votes
2answers
163 views

Deep neural net modelling strategy

are there any resources out there (book, blog, your own answer post etc.) that gives advise on modelling strategy of deep neural net? I know how to fit a neural net, I know how to change settings ...
1
vote
0answers
509 views

Why is this trained model misclassifying new cases?

Below is text / code from a recent SO post of mine that I hope you could comment on: Using Keras/Theano, I successfully trained a model using the Keras sample code for CIFAR10. It ran for about 2 ...
4
votes
2answers
2k views

How to handle Memory issues in training Word Embeddings on Large Datasets?

I want to train a word predictability task to generate word embeddings. The document collection contains 243k documents. The code implementation is in torch. I am struggling with the huge size of the ...
4
votes
1answer
4k views

Google TPU: when/how will it be available to me?

Google recently announced the TPU custom chip. They stated it is available in Google Cloud Platform, but only for their internal usage. When will I be able to use it? Will it be possible to buy this ...
1
vote
0answers
297 views

Multi-GPU, multi-machine computation with Torch

I know that Torch supports multi-GPU computation on the same machine (Example). Is it possible to perform multi-GPU, multi-machine computations with Torch?
1
vote
1answer
639 views

LeNet for Convolution network?

I keep seeing LeNet used to referring to a convolution network? I am wondering why LeNet is called LeNet? Is it the abbreviation of anything? Is there a difference between LeNet and convolutional ...
2
votes
1answer
481 views

Difference between Validation data and Testing data?

I am bit confused about validating data. What is this data mainly for?? Like I am seeing some tutorial and they have some training[I know it] images , they they have some validation images[donot know] ...
0
votes
2answers
1k views

cifar10 official keras example not giving expected accuracy, using sigmoid seems better than relu

In the official Keras example cifar10 there is the following code to train a CNN using keras10. When I tried it, my neural net would not learn at all, I always get around a 10% acuracy, which is ...
10
votes
2answers
22k views

Question about bias in Convolutional Networks

I am trying to figure out how many weights and biases are needed for CNN. Say I have a (3, 32, 32)-image and want to apply a (32, 5, 5)-filter. For each feature map I have 5x5 weights, so I should ...
2
votes
1answer
185 views

Trying to figure out how to set weights for convolutional networks

I am working on CNN, and I have some doubts. Let's assume I only want one feature map, just to make things easier. And let's suppose my image is grayscale, to make things even easier. So, let's say my ...
2
votes
1answer
125 views

Applying ConvNets to classify motion/video data

How would someone go about using deep learning to classify sign language gestures? For example, suppose I had video files of many different gestures. For any given gesture, I might have many videos of ...
5
votes
2answers
236 views

How to predict on part of image after training on other part of image?

I have images of identity cards (manually taken so not of same size) and I need to extract the text in it. I used tesseract to predict bounding boxes for each letter and am successful to some extent ...
2
votes
0answers
147 views

In plain English, how to descibe i/o of the TensorFlow for language modelling?

I have followed the tutorial here about language modelling using Tensorflow to create LSTM and used PTB dataset. The code is here I failed to understnad the exact specific input and the output of the ...
5
votes
2answers
29k views

Cross Validation in Keras

Suppose I would like to train and test the MNIST dataset in Keras. The required data can be loaded as follows: ...
3
votes
3answers
2k views

Predicting next action to take to reach a final state

Does anyone know of an algorithm that could be used to determine the next action to take to reach a desired state when trained on time-series data? For example, a robot starts at a certain state, ...
10
votes
1answer
19k views

Reshaping of data for deep learning using Keras

I am a beginner to Keras and I have started with the MNIST example to understand how the library actually works. The code snippet of MNIST problem in Keras example folder is given as : ...
6
votes
4answers
1k views

What GPU specifications matter when training and using neural networks?

I need to purchase some GPUs, which I plan to use for training and using some neural networks (most likely with Theano and Torch). Which GPU specifications should I pay attention to? E.g.: one ...
10
votes
2answers
1k views

Relu does have 0 gradient by definition, then why gradient vanish is not a problem for x < 0?

By definition, Relu is max(0,f(x)). Then its gradient is defined as: 1 if x > 0 and 0 if x < 0. Wouldn't this mean the ...
8
votes
1answer
601 views

Is there any domain where Spiking Neural Networks outperform other algorithms (non-spiking)?

I'm reading about reservoir computing techniques like Echo State Networks and Liquid State Machines. Both of the methods involve feeding inputs to a population of randomly (or not) connected spiking ...
5
votes
1answer
380 views

Solving multi label image classification using TimeDistributed dense layer

I have a multi label image dataset having 5 labels. Each image can have more than one label at the same time. I am using a convolutional neural network to extract features and those extracted features ...
3
votes
1answer
366 views

Noisy behavior in deep learning feed-forward net

I am a bit unsure about optimizing a neural net with 3 or more layers. The input data is quite noisy and I seem to project the noise into the learning (strong bias in the data, 90% belong to one class ...
4
votes
1answer
132 views

Training neural nets: is it important that the data is randomly sorted?

Let's say I have a neural network that classifies data into A, B or C. I've heard that it's bad to train the network with data from one class at a time, e.g. ...