Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
2
votes
1answer
494 views

Variational auto-encoders (VAE): why the random sample?

Why do people train variational auto-encoders (VAE) to encode means and variances (regularised towards 0 and 1), and then sample a random Gaussian, rather that simply encode latent vectors and ...
0
votes
1answer
245 views

Non trainable problems

Before facing this question, I always thought non-learnable problems are those which the provided data for the problem has high amount of outliers, those which don't have sufficient features or those ...
11
votes
1answer
3k views

Can BERT do the next-word-predict task?

As BERT is bidirectional (uses bi-directional transformer), is it possible to use it for the next-word-predict task? If yes, what needs to be tweaked?
9
votes
1answer
4k views

Understanding dropout and gradient descent

I am looking at how to implement dropout on deep neural network, and I found something counter intuitive. In the forward phase dropout mask activations with a random tensor of 1s and 0s to force net ...
9
votes
1answer
609 views

What are “VGG54” and “VGG22” derived from the VGG19 CNN?

In the paper Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network by Christian Ledig et al., the distance between images (used in the loss function) is calculated from ...
6
votes
2answers
6k views

Binary Classifier making only one prediction

tl;dr I'm building a binary classifier that always eventually predicts all "0" or all "1" after some number of epochs and I'm looking for possible reasons why/how to proceed. Below is all just more ...
6
votes
4answers
289 views

Learning capacity: Deep Learning vs Traditional (Shallow) Learning

I am currently doing a course in coursera in which Andrew Ng draws the following image: Does anybody know any references/reasoning that justify the drawn graph? Were any experiments conducted to ...
5
votes
1answer
1k views

Decision Tree used for Calculating Precision, Accuracy, and Recall, class breakdown question

I am creating decision trees modeling data that looks like this. ...
5
votes
2answers
4k views

Using cross-validation technique for a CNN model?

I am working on the CNN model, as always I use batches with epochs to train my model, for my model, when it completed training and validation, finally I use a test set to measure the model performance ...
5
votes
1answer
3k views

Convolution and Cross Correlation in CNN

What would be the intuition behind using the convolution and cross correlation operation inside Convolutional Neural Networks? I am interested in putting together the theory from Digital Image ...
4
votes
3answers
5k views

Number of Fully connected layers in standard CNNs

I have a question targeting some basics of CNN. I came across various CNN networks like AlexNet, GoogLeNet and LeNet. I read at a lot of places that AlexNet has 3 Fully Connected layers with 4096, ...
4
votes
1answer
1k views

Why do pre-trained CNNs use low image resolution?

I want to use a pre-trained convolutional network for image classification. My base data has resolutions of 500x500px up to 1000x1000px. Pre-trained architectures often expect less (between 255 and ...
4
votes
1answer
112 views

Can CBOW model only accept fixed number of words?

I have a question about CBOW prediction. Suppose my job is to use 3 surrounding words w(t-3), w(t-2), w(t-1)as input to predict one target word w(t). Once the model is trained and I want to predict a ...
4
votes
1answer
868 views

What is deconvolution operation used in Fully Convolutional Neural Networks?

When I was reading this this paper, Fully Convolutional Networks for Semantic Segmentation, I found that they use an up-sampling layer to classify each pixel in to a class. I have two questions: How ...
4
votes
2answers
4k views

Should there be a flat layer in between the conv layers and dense layer in YOLO?

Should there be a flat layer in between the conv layers and dense layer in YOLO? It's something not specified in the paper, but I see most implementations of YOLO on github do this. In my ...
4
votes
2answers
3k views

What are the cases where it is fine to initialize all weights to zero

I've taken a few online courses in machine learning, and in general, the advice has been to choose random weights for a neural network to ensure that your neurons don't all learn the same thing, ...
3
votes
3answers
1k views

Evaluation methods for multi-class classification

I am looking for single-number evaluation method that can be used in multi-class classification tasks that take into account imbalanced data-sets. For instance, ...
3
votes
1answer
147 views

Using deep-learning on graph data for binary classification

The data: I have certain data that I decided to represent it as a graph (I thought it would suit). So I have the weighted graph data that includes a numeric attribute for each node. (networkx graphs)....
3
votes
1answer
883 views

Probability of dropout growth

In the DNN literature, is there analysis or a term on a dropout ratio (oppositely-)proportional to the depth of a layer? By intuition, I'd like to dropout fewer neurons on the layers next to the ...
3
votes
2answers
1k views

What are the possible approaches to fixing Overfitting on a CNN?

Currently I am trying to make a cnn that would allow for age detection on facial images. My dataset has the following shape where the images are grayscale. ...
3
votes
1answer
8k views

Number of parameters in an RNN

I'm using a basic RNN as in the figure below (say for translation). The model has the following structure: \begin{aligned} s_t &= \tanh(Ux_t + Ws_{t-1}) \\ o_t &= \mathrm{softmax}(Vs_t) \end{...
3
votes
0answers
893 views

Multivariate, multistep forecasting with LSTM

I want to use an RNN with LSTM to forecast multiple steps into the future, based on multiple inputs. I have some ideas for different ways to approach this, but I'm afraid I'm missing the "right way" ...
3
votes
1answer
2k views

Activation method and Loss function for multilabel multiclass classification

I am using CNN for Sentence Classification code by Yoonkim. This is used for text classification. I noticed that he uses softmax layer and negative log likelihood error. This is optimal for single ...
2
votes
1answer
206 views

What is wrong with this reinforcement learning environment ?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
2
votes
1answer
955 views

How hidden layer is made binary in Restricted Boltzmann Machine (RBM)?

In RBM, in the positive phase for updating the hidden layer(which should also be binary), [Acually consider a node of h1 ∈ H(hidden layer vector)] to make h1 a binary number we compute the probability ...
2
votes
3answers
3k views

How to do spelling correction for a language but also correct some words in other language

I want to do spell correction for the portuguese language, specifically for restaurant bots. The problem is that food names aren't always in portuguese as well, and for that reason are the most likely ...
2
votes
0answers
139 views

How to resolve the instability of average reward per episode in training of DQN (Deep Q-Network)?

what is shown when average reward per episode in training is unstable? If there is big difference between average reward per episode and final reward by test section, what we can say? For ...
2
votes
0answers
58 views

intercept correction in deep learning

Say I have an imbalanced data set, and I decided to over/undersample it during model training. I would then like to predict on new records but using the original, true imbalance in the classes as an ...
2
votes
1answer
47 views

How to Bootstrap dataset for 10000 AUC scores?

I am new to ML and trying to learn the nuances. I work on a binary classification problem with 5K records. Label 1 is 1554 and Label 0 is 3554. What I currently do is 1) split the data into train(...
1
vote
1answer
670 views

Estimating Predictive Uncertainty for unlabeled data

I am trying to estimate the predictive uncertainty for a deep neural network. While I do have a labeled training set, I´m trying to measure uncertainty for some unlabeled production data. This paper ...
1
vote
0answers
352 views

Ground truth/label modification during training (with the data obtained from the

I'm working on an image segmentation algorithm with FCN (Long et al., 2015) as the backbone network. One idea I have is to use the argmax binary mask obtained from the final score layer (250x250x1) ...
1
vote
1answer
2k views

Build knowledge bot using deep learning

I'm going through this chatbot example, which uses the Cornell movie dialog corpus. Expanding this example, is it possible to build a "knowledge bot" (ie) a bot that can chat and be knowledgeable in a ...
1
vote
2answers
1k views

Adding hand-crafted features to a convolutional neural network (CNN) in TensorFlow

Let's say I want to add a few hand-crafted features to a convolutional neural network CNN in TensorFlow. The CNN can be a simple one as described here. Naturally I'd like to add these features right ...
0
votes
1answer
5k views

autoencoder for features selection [closed]

I m using a data set with 41 features numerics and nominals the 42 one is the class (normal or not) first I changed all the nominals features to numeric since the autoencoder requires that the imput ...
6
votes
3answers
3k views

What is normalization for?

I am new in python and data science (and not great in math). I am learning machine learning. I got following normalize function. Can you please explain what does this normalize function do? ...
5
votes
2answers
5k views

What is the different between Fine-tuning and Transfer-learning?

Usually the neural network training has at least 2 steps: first trained on a large set of some standard data (ImageNet, ...) and then the resulting weights are trained on a small set of my data (in ...
4
votes
1answer
6k views

TensorFlow: number of channels of conv1d filter

I want to apply a ConvNet on my one dimensional data retrieved from 13 sensors. So, each of my samples consists of 13 channels (of 51 values) I am using 'conv1d' ...
4
votes
2answers
66 views

Discrimination vs Calibration - Machine Learning Models

I came across a new term called Calibration while reading about prediction models. Can you please help me understand how different it is from ...
4
votes
1answer
358 views

Meaning of dropout

What does model.add(Dropout(0.4)) mean in Keras? Does it mean ignoring 40% of the neurons in the Neural Network? OR Does it mean ignoring the neurons that give ...
4
votes
3answers
4k views

What cost function and penalty are suitable for imbalanced datasets?

For an imbalanced data set, is it better to choose an L1 or L2 regularization? Is there a cost function more suitable for imbalanced datasets to improve the model score (...
4
votes
1answer
46 views

How to make overfitting (powerful) model?

According to my professor one of the first steps in modelling a NN is to use a powerful enough model. The first step is to create a model that is powerful enough to achieve very high accuracies (...
3
votes
2answers
773 views

What is the state-of-the art ANN architecture for MNIST?

What is actually the best neural network architecture for the classic MNIST digit classifying task? I couldn't find any that would claim to be the winner...
3
votes
1answer
88 views

Is it a red flag that increasing the number of parameters makes the model less able to overfit small amounts of data?

I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~...
3
votes
1answer
897 views

Which is the fastest image pretrained model?

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster ...
3
votes
1answer
400 views

How to design batches in a stateful RNN

I am using TF Eager to train a stateful RNN (GRU). I have several variable length time sequences about 1 minute long which I split into windows of length 1s. In TF Eager, like in Keras, if ...
3
votes
1answer
912 views

Number of parameters in Neural Network Language Model

The Neural Network language Model (NNLM) by Bengio et.al is a structure extensively used in machine translation, text summarization based on deep learning. What's the computational complexity of this ...
3
votes
1answer
808 views

How to apply my deep learning model to a new dataset?

I am doing semantic segmentation (multi-class classification of image pixels) using convolutional neural networks (CNN) in Keras. In particular, I am applying this to aerial images of crops (...
2
votes
1answer
183 views

Depth of the first pooling layer outcome in tensorflow documentation

Let's say that we have a CNN with two convolutional layers (https://www.tensorflow.org/tutorials/layers). My question regards the dimension of the tensor, which is the output of the pooling layer 1. ...
2
votes
1answer
247 views

CNN training data size for determing the winner of tic-tac-toe

I'm trying to learn machine learning with tensorflow and wrote a program that uses CNNs to determine game results for a given tic-tac-toe board. Its inputs and outputs are - Input - An array of 9 ...
2
votes
2answers
869 views

Data augmentation based on the class type in the CNN model

I would like to use CNN model to classify images but some classes in my dataset have low amount of data. Can I apply data augmentation based on the number of the images in the class? For example, ...