Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
7
votes
1answer
5k views

Why do we need to add START <s> + END </s> symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?

In the Sequence-to-Sequence models, we often see that the START (e.g. <s>) and END (e.g. </s>) symbols are added to ...
3
votes
1answer
7k views

How to initialize word-embeddings for Out of Vocabulary Word?

I am trying to use CoNLL-2003 NER (English) Dataset and I am trying to utilize pretrained embeddings for it. I am using SENNA pretrained embeddings. Now I have around 20k words in my vocabulary and ...
2
votes
1answer
11k views

“concat” mode can only merge layers with matching output shapes except for the concat axis

I have a function I am trying to debug which is yielding the following error message: ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer ...
2
votes
1answer
779 views

If deep-learning learns features, aren't we saying it can learn association rules?

If deep-learning learns features, aren't we saying it can learn association rules ?
1
vote
1answer
1k views

LSTM Implementation using tensorflow (anaconda)

I'm new to TensorFlow and currently I'm trying to implement an LSTM using jupyter notebook. But when I run the following code segment, I got some errors and couldn'...
4
votes
2answers
1k views

Multilabel image classification: is it necessary to have traning data for each combination of labels?

I want to train a CNN for a multilabel image classification task using keras. However I am not sure how to prepare my tranining data. More specifically, I am wondering if I need training images that ...
8
votes
1answer
6k views

Data preprocessing: Should we normalise images pixel-wise?

Let me present you with a toy example and a reasoning on image normalisation I had: Suppose we have a CNN architecture to classify NxN grayscale images in two categories. Pixel values range from 0 (...
2
votes
1answer
319 views

Compute backpropagation

I have the question which is mentioned in the above picture. It is trying to find the derivative of f with respect to weight matrix ...
1
vote
1answer
917 views

Training error is oscillating, cross validation error is continuously decreasing

I created a two layered fully connected neural network as a part of a recommendation engine (after I use embedding layers for products and users). I have been trying to tune the hyper-parameters for ...
3
votes
2answers
5k views

What are useful evaluation metrics used in machine learning

I am using CNN in order to predict codes after analyzing text. As an example, I will write "I am crazy" .. the model will predict some code " X321". All this based on CNN. I want to evaluate my ...
1
vote
1answer
605 views

Which Classification Metrics Are Appropriate For Each Class Distribution Scenario?

Currently, I have a balanced dataset (that I artificially over-sampled to make it balanced). My classes are binary (0 or 1). I'm wondering if "accuracy" is the "best" metric to use in the situation ...
1
vote
1answer
1k views

What loss function should I use if I have been working on a classification problem which involves both multi-label and multi-class labels?

For example I have apple and pear pictures. What I am trying to do is to predict if a picture is an apple or pear picture and AT THE SAME TIME predicting whether the fruit is big and/or yellow. Thus ...
-1
votes
4answers
390 views

CNN'S are what?

I have a very fundamental question on what CNN'S actually are. I understand fully the training process as to take a bunch of images, start with random filters, convolve, activate, calculate loss, back ...
2
votes
2answers
117 views

Should the different layers of deep learning models have same size or they should be changed based on a rule

I see a lot of people varying the width of each layer in a deep neural network. ie. Input->50->100->150->Output. I'm curious what, if any, are the advantages of this structure over static layer widths ...
14
votes
2answers
9k views

Is there away to change the metric used by the Early Stopping callback in Keras?

When using the Early Stopping callback in Keras training stops when some metric (usually validation loss) is not increasing. Is there a way to use another metric (like precision, recall, f-measure) ...
3
votes
0answers
4k views

Custom layer in keras with multiple input and multiple output

I have implemented a custom layer in keras which takes in multiple input and also results to multiple output shape. My code goes as below: ...
0
votes
1answer
7k views

Keras - Computing cosine similarity matrix of two 3D tensors

Using TF backend, I need to construct a similarity matrices of two 3D vectors, both with shape (batch_size, N, M), being N and M natural numbers. The function ...
1
vote
0answers
35 views

Deep learning: parameter selection and result interpretation

I have a multiclass(7 labels) classification problem implemented in MLP. I am trying to classify 7 cancers based on some data. The overall accuracy is quite low, around 58%. However, some of the ...
3
votes
1answer
110 views

Localised filters for CNNs

I was going through some material and the term "localized filters" was used, can someone please explain what it means and what are the advantages of having a localized filter. EDIT-1 My question is ...
1
vote
0answers
26 views

How to interpret long equations in Deep Learning papers

For eg. I've been studying a paper on Recommender systems using collaborative deep learning and I've just started learning. The paper revolves around the NN representation as shown below The ...
5
votes
2answers
2k views

Does it make sense to combine PCA with an artificial neural network?

I have a Dataset of around 200 features. Most of them are categorical and only a few are numerical. It seems that an artificial neural network with an Autoencoder has some problems with that kind and ...
1
vote
1answer
40 views

Can't interpret the text information and ratings matrix imported to NN

I have a Recommender system which uses a Collaborative bayesian approach using pSDAE for recommending scientific articles from the Citeulike Dataset The text information (as input to pSDAE) is in the ...
2
votes
1answer
271 views

multi channel feature classification using deep feed forward neural network on tensorflow [closed]

Can any one suggest a dataset that contains multi channel/multi dimensional feature values that can be classified using a deep feed forward neural network? Please ask me if the question is not clear ...
1
vote
1answer
162 views

Understanding why in deep reinforcement learning correlations in the data reduce the effectiveness

From the paper Human-level control through deep reinforcement learning, Mnih et al. Nature 2015 It says ...
5
votes
3answers
4k views

Neural Network Performs Bad On MNIST

I've been struggling with Neural Networks for a while now. I get the math behind backpropagation. Still as reference I'm using the formulas from here. The Network learns XOR: Prediction After ...
29
votes
6answers
5k views

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide ...
4
votes
1answer
343 views

How do neural networks account for outliers?

How do neural networks account for outliers and overfitting?
1
vote
0answers
64 views

How can we decompose generalization gap as done in the paper “Generalization in Deep Learning”?

I am reading a recent paper "Generalization in Deep Learning" and I am unable to understand a step. In this step, they first take neural network as a direct acyclic graph(DAG) and described output of ...
1
vote
0answers
22 views

Data set for Q & A system training [closed]

What is the standard (or just good) open data set to train a model for question answering? Ideally, not only in English. Thanks!
1
vote
1answer
3k views

Backpropagation with multiple different activation functions

How does back-propagation handle multiple different activation functions? For example in a neural network of 3 hidden layers, each with a separate activation function such as tanh, sigmoid and ReLU, ...
1
vote
0answers
110 views

loss = function(iteration) gets super wobbly once it gets near the bottom

When fitting neural nets and getting close to the bottom, I consistently get a very distinct pattern in the loss function (and the mse). See below: (The lower right plots just give the last 100 ...
3
votes
1answer
4k views

Is there a maximum limit to the number of features in a Neural Network?

I have created a dataset which has rather large number of features for example-100,000. Is it too large for a decent computer to handle ( I have a 1080ti )?
3
votes
2answers
183 views

What ML/DL techniques power Youtube/Netflix search systems?

Video platforms like YouTube, Netflix, Amazon prime have an excellent search system - given a search string, find most relevant videos. Which Machine Learning /Deep Learning techniques used for this? ...
1
vote
1answer
119 views

Doing a fine tuning after a transfer learning

I red about fine tuning and transfer learning for CNNs and I was wondering if we can do fine tuning after using transfer learning on the same CNN , if so will this increase the performance of the ...
27
votes
3answers
16k views

Why is ReLU used as an activation function?

Activation functions are used to introduce non-linearities in the linear output of the type w * x + b in a neural network. Which I am able to understand ...
2
votes
1answer
209 views

In CS231n lecture, can't the linear classifier be softmax itself?

I am little bit confused on why the scoring function that is the $f(X,W)$ is chosen to be $W,X$ while they talk about Softmax and SVM loss in this. Can't they take Softmax classifier or SVM ...
1
vote
0answers
155 views

Attention network over text dataset

I am implementing an attention network model over a categorical CQA dataset. Following is my code for the model: ...
5
votes
1answer
194 views

Suspected Exploding Gradient in Character Generator LSTM

I'm trying to create a neural network that can learn how to write text character by character from the book David Copperfield (via Project Gutenburg). It starts great, then forgets punctuation ...
1
vote
1answer
2k views

correct value of output_dim or units parameter of dense

I have started learning deep learning and using keras library. But I am confused as to how to take a proper estimate of the value to use for ...
4
votes
2answers
6k views

What is the shape of conv3d and conv3d_transpose?

I want to do a GAN with coloured pictures. This means I need a three dimensional input and therefore I like to use conv3d and conv3d_transpose. Unfortunately in the TensorFlow documentation, I can't ...
4
votes
4answers
6k views

Feature Scaling of Training Set and Test Set

Suppose I want to use the Gradient Descent algorithm. I have a training set and a test set and I want to do the feature scaling with mean normalization. Should I use the same mean and variance for ...
4
votes
1answer
1k views

Factors of Variation in Deep Learning

In the textbook im reading about Deep Learning, i found : When designing features or algorithms for learning features, our goal is usually to separate the factors of variation that explain the ...
1
vote
1answer
103 views

Gradient Exchange

I read a paper on Deep Neural Networks Compression (link: https://openreview.net/forum?id=SkhQHMW0W) and came across a term "gradient exchange", I tried making sense out of it but couldn't exactly ...
2
votes
2answers
2k views

How implement a feed forward network with arbitrary node connections in keras?

I'm new to using the keras framework. I have read some examples about how to construct deep learning models with the Sequential and Graph classes in keras. However, I see that, independently if I use ...
2
votes
0answers
705 views

Re-scale data after PCA for an LSTM?

I want to use the result of my PCA as an input for my LSTM model. I began by Applying the MinMaxScaler and then did the PCA, (then I reshaped my data of course) : ...
3
votes
1answer
3k views

Example of 1D ConvNet filter

I understand Conv2D filters. I think I understand Conv1D filters as well but have not seen any examples of the filters like what ...
2
votes
1answer
66 views

After a Feature Scaling do i have the same initial information?

I'm studying the gradient descent algorithm for single hidden layer neural networks. Suppose that I have an initial dataset and then I use mean normalization in order to scale the features. Why ...
0
votes
1answer
69 views

Does self driving technology gain more from the data or the state-of-the-art algorithm?

When people start to figure the self driving cars will replace some vehicles on the road in the very near future, it somehow means the learner in the robot software could reach a very low empirical ...
1
vote
1answer
72 views

A neural network that keeps learning as new training data becomes available

I have a text-classification problem with a lot of training data. Running cross-validation takes a lot of time - several days or even weeks. In order to make the system more responsive, I am thinking ...
2
votes
1answer
759 views

Transfer learning (on pre-trained inception net model) for multi label classification is giving similar probability for all labels

Number of labels: 1000, Dataset size: 200000 images Final probability for 1000 labels is in the range of 0.3 to 0.34. I was expecting large variation in probabilities. Can someone tell me what I am ...

1
50 51
52
53 54
65