Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
1
vote
1answer
32 views

Normalize the output of a dense layer with linear activation

I have the following architecture of my network: ...
0
votes
1answer
82 views

ReLU for combating the problem of vanishing gradient in RNN?

For solving the problem of vanishing gradients in feedforward neural networks, ReLU activation function can be used. When we talk about solving the vanishing gradient problem in RNN, we use a more ...
0
votes
0answers
65 views

Building a speech commands dataset for audio recognition applications

I'm working on a DL project to recognize (10 - 15) Arabic speech commands from a continuous stream of audio, and I want to create a dataset similar to Google's Speech Commands dataset. Fortunately, I ...
0
votes
0answers
9 views

play session from ratings dataset in Movie20 M

I need to extract listening sessions from the ratings dataset which has the columns cols = [userId movieId rating timestamp] timestamp is just a number for eg 1112486027 listening sessions are ...
0
votes
0answers
9 views

Will an object detector detect all the 10 objects in an image if the dataset had 1 object per image (the images had 10 similar objects)?

I was making an object detector which will detect balloons in an image. now since there were lot of balloons in the images of the dataset. so I annotated only 1 or 2 balloons per image even if there ...
0
votes
0answers
15 views

Scaling ML/DL classifier

I have been trying to find some guideline through google/stackoverflow for scaling a classification system. E.g. how can I scale a face recognition system if we want to add new people into the system? ...
0
votes
0answers
23 views

nlp: Translation System: Transformer/GPT2 model: Why do we need to mask future tokens?

I am trying to understand the whole concept of masking the tokens in the transformer/gpt2 model. In this blog post, http://jalammar.github.io/illustrated-gpt2/ the author takes an example where " the ...
0
votes
0answers
11 views

Does a CNN fully memorize ground truth if it has more parameters than training pixels?

ResNet consists of 25M trainable parameters. If only 30% of 600 $512 \times 512$ images is annotated, there are $600 * 512 * 512 * ~0.3 = 47,185,920$ ground truth pixels. A parameter is a floating ...
1
vote
4answers
62 views

What is the intuition behind larger number of samples are better for statistics?

It may be well-known that when we take statistics, we essentially need a large number of samples. Because I am taught this fact before studying the math, I have been here without exploring the reason. ...
1
vote
1answer
2k views

Distractor Generation for Multiple Choice Questions

I'm currently working on generating distractor for multiple choice questions. Training set consists of question, answer and 3 distractor and I need to predict 3 distractor for test set. I have gone ...
0
votes
1answer
12 views

``Hierarchial features extraction'' in Multilayer Perceptron models

I am referring to plain neural networks, MLPs. I got to read the paper by Glorot and Bengio (2010), Understanding the difficulty of training deep feedforward neural networks. Therein I read an ...
8
votes
4answers
246 views

Math PhD (Nonlinear Programming) switching to Data Science?

I am a math Ph.D. student who is interested in going to the industry as a Data Scientist after graduation. I will briefly give some background on my education before posing my question, so that it is ...
0
votes
1answer
31 views

The loss and accuracy of this LSTM both drop to nearly 0 at the same epoch

I'm trying to train an LSTM to predict the the Nth token using the N-1 tokens preceding it For each One-Hot encoded token, I ...
0
votes
0answers
25 views

Spatial and temporal information processing together (CNN and LSTM)

I have small problem that requires to process both spatial and temporal information. I need to predict vehicle's trajectory based on previous trajectory information and map information. My current ...
0
votes
0answers
19 views

Proof that gradient descent takes exponential time for escaping saddle points

https://arxiv.org/pdf/1705.10412.pdf I was going through this paper, and understood the crux of it. But in appendix, the complete proof of it is given which was a bit tough for me mathematically. So ...
0
votes
0answers
18 views

How to use list of numpy arrays to train ML algorithm?

I'm trying to develop a machine learning algorithm using LinearSVC and another one using Convolutional Neural Networks to classify DNA sequences. I've had to one hot encode the DNA sequences and then ...
0
votes
0answers
6 views

How to draw a support set when classifying using Siamese networks without performing one shot learning?

How to perform classification on a test set with Siamese networks when I cannot afford to draw the support set from the test set itself? Possible options which come to my mind are: KNN using samples ...
0
votes
0answers
68 views

How can we use resnet in autoencoders?

I am creating an unsupervised classifier model, for which i want to use resnet 50 on a custom database and used the top layers of resnet as start point of my autoencoder. How to proceed with this. ...
0
votes
0answers
9 views

How Cycle GAN translates between very different objects?

I'm trying to understand how the popular CycleGAN responds if the objects to be translated between are very different (horse and map or house and apples). All of the examples appear to be translating ...
2
votes
1answer
45 views

What is auxiliary loss in Character-level Transformer model?

I am reading Character-Level Language Modeling with Deeper Self-Attention from Rami Al-Rfou. In the second page, they had mentioned about Auxiliary Losses which can speed-up the model convergence and ...
0
votes
1answer
16 views

Training CNN for Regression

Background: I am using CNN to predict forces acting on a circular particle in a granular medium. Based on the magnitude of the forces, particle exhibits different patterns on its surface. The images ...
0
votes
0answers
15 views

Will it be more computationally expensive to have multipl 2d tensors or 1 3d tensor

Odd question but I am busy creating a Genetic Algorithm that optimizes the weights on a Neural Network instead of using good old fashion 1st-order optimization (Gradient/Adam) What I have is x as a ...
1
vote
1answer
42 views

Predict correct answer among ten answers for a given question

I have a case study to solve where I am given a dataset of questions and its answers, there are ten answers for a particular question. It's a classification problem where correct answer is having <...
1
vote
1answer
35 views

Neural Networks: Predicting probabilities of the possible values of y, instead of just predicting y

I have a true value y that I'd like to predict with a regression, but I'm interested in the probabilities that y will be different values. Y is theoretically continuous but in the dataset it is ...
0
votes
0answers
16 views

How to calculate Temperature variable in softmax(boltzmann) exploration

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...
2
votes
0answers
22 views

How can RNN handle variable sized inputs?

I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? here Let us suppose we want to do a ...
1
vote
0answers
14 views

temperature variable in boltzmmann-exploration in reinforcement learning

I have been using epsilon greedy action selection strategy and recently have come across boltzmann(softmax) action selection strategy. One thing I am not clear about boltzmann exploration is the ...
0
votes
0answers
11 views

what will happen if by mistake i train a object detection algorithm with images containing multiple bounding box in the same object?

I have a dataset of images where I may have some images where the bounding box is annotated time on the same object. Will that create a problem in the accuracy of the model?
0
votes
0answers
9 views

Scenario description of Roads using Deep learning

I want to do scenario description of road. If there are cars, humans infront of the blind person it should describe the scenario and output should be consumable by blind person. Question: Should it ...
0
votes
0answers
22 views

Representing and Training Individualized Models

Say I want to create a handwriting OCR or speech-to-text system intended for many users. A first pass might be to train a single one-size-fits-all model on all available data to predict all users' ...
0
votes
1answer
27 views

Padding the sentences is consuming huge memory

I prepared a lstm model using tensorflow which has a max_sequence_length of 5000 and I'm padding the small sentences with 0's. I then deployed and tested the model ...
1
vote
1answer
141 views

Machine learning method to predict event date

Let's say I have a big dataset consisting of variables including but not limited to the start/end date of loans, their notional amount, a loan prepayment indicator etc. My goal is to create a model ...
0
votes
1answer
14 views

What is the State of the art method for full body gesture recognition in images

I am working on gesture recognition in images and the best way that I am aware of, is whether using end to end approaches with deep neural networks or extracting body joint positions in an image and ...
0
votes
2answers
54 views

How to normalize a data set of multiple time series?

I have the a data set representing the electricity consumption of 25 000 customer. The electricity readings are taken from each smart meter each 15 min for a period of 3 days. The data is takes from ...
0
votes
1answer
24 views

Explanation behind the calculation of training loss in deep learning model

I am trying to model an image classification problem using convolution neural network. I came across a code on Github in which I am not able to understand the meaning of following line for loss ...
0
votes
0answers
6 views

SRGAN: How to adapt the model to the input image?

I wrote and trained my own SRGAN: so I obtained a generator’s model that takes 32x32 images as input and gives their improved 128x128 version as output… However, the end users of my Android app will ...
0
votes
1answer
31 views

Autoencoder or layer-based dimensionality reduction?

I have a few TB of wide data. I want to reduce the number of features in my dataset before feeding my dataset into a classification model... or should I not? Obviously, I will want to try both ...
0
votes
2answers
19 views

Choosing a set of CNNs for paper

There are so many CNNs out there and im trying to do a comparison between some of them in my paper which networks should I use? Resnet, vgg and inception are obvious but I need 3 or 4 others. which ...
0
votes
0answers
48 views

How to prepare coordinate sequence data for machine learning classification?

I want to perform a task where the goal is to classify coordinate sequences by labels. The raw data consists of temporal log sequences for each label like this: ...
0
votes
0answers
20 views

Matrix multiplication doesn't work - no output

I am having problem in the Matrix multiplication of my Python neural network. Being still a High schooler, I know next to nothing about MM except a couple of tutorials. My Neural network was working ...
1
vote
0answers
87 views

What is weighted cross entropy loss?

I'm reading Cliche, Mathieu (2017). In his paper, he describes using cross entropy loss, weighted by the inverse frequency of the true classes to counteract an imbalanced dataset (this is stated on ...
0
votes
0answers
10 views
0
votes
1answer
38 views

How does Neural Network denoise an image?

I understand the mathematical formalism behind how neural networks work as a classifier or perform regression analysis. But I face difficulty to realize how they are such a great denoising instrument. ...
0
votes
0answers
28 views

How to get same accuracy with identical models in Keras and Tensorflow?

As we all know Keras backend uses Tensorflow and so it should give out same kind of results when we provide same parameters, hyper-parameters, weights and biases initialisation at each layer, but ...
0
votes
0answers
11 views

Problem with convolution neural network gradient checking

I have implemented a deep learning network : Conv -> Relu -> Maxpool -> flattens -> dense -> softmax. the network has 6178 parameters. I am trying to do gradient checking on my deep learning network. ...
0
votes
1answer
48 views

How to get Keras accuracy for each step in an epoch like in Tensorflow?

Like in tensorflow I get accuracy for each step - ...
1
vote
0answers
84 views

Tensorflow and Keras model having same parameters, hyperparameters, weight and bias initialization give different accuracy?

I have made sure that layers,parameters, hyperparameters,kernel_initialization, bias_initialization, seed and dataset are all equal. But still the output for both the models are different. ...
1
vote
2answers
142 views

How can I get probabilities of next word with ELMO?

ELMO is a language model, build to to compute the probability of a word, given some prior history of words seen. How can I get this probability from pretained ELMO model?
0
votes
2answers
82 views

What's the proper way to do back propagation in Deep Fully Connected Neural Network for binary classification

I tried to implement a Deep fully connected neural network for binary classification using python and numpy and used Gradient Descent as optimization algorithm. ...
0
votes
0answers
12 views

Creating a map with camera using Lidar data

I want to create a neural network that learns from lidar data to create a map from image by using camera. I want to start but I need some advices to proceed. I will collect data from lidar and I want ...