Questions tagged [deep-network]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
4
votes
1answer
35 views

What are some (non image) datasets where deep networks are required for good performance?

It is well known that deep networks tend to outperform shallow networks and other classical machine learning techniques such as boosting on learning tasks involving images. I believe this is because ...
0
votes
0answers
14 views

How to calculate the memory usage of a deep LSTM network?

I was trying to estimate the memory usage for my LSTM network by referring to an examples of CNN memory usage calculation at http://cs231n.github.io/convolutional-networks/#computational-...
0
votes
1answer
7 views

Apply LSTM to each matrix element with Keras

I'm trying to apply a LSTM/GRU to each entry of a matrix $X$ note: Each matrix element is a time-series, so shape of X is (batch_size, rows, cols, time_steps, dims) $ y_{i,j}= \begin{cases} ...
0
votes
0answers
11 views

How to change classification model architecture for a new target application

I'm new to Deep Learning with Keras. With some tutorials online for cat vs non-cat classification, I was able to compile this simple architecture for my own ...
1
vote
1answer
37 views

Learning rate Scheduler

A very important aspect in deep learning is the learning rate. Can someone tell me, how to initialize the lr and how to choose ...
0
votes
0answers
11 views

NL2SQL task, if we have enough data, what will the model achieve for hard SQL?

We are afraid that the hard SQL like TABLE JOIN is the limit for industrial application. Addition info: https://yale-lily.github.io/spider Thank you very much.
0
votes
0answers
15 views

What will happen if we replace the transformer of BERT to evolved transformer?

If we replace the official BERT's transformer to evolved transformer, do the change accelerate the inference speed without losing accuracy?
1
vote
1answer
23 views

When would not normalizing input values have higher accuracy?

Right now I'm training a deep neural network for a binary classification problem, with a feature set of winrates. As such, each winrate is bigger or equal to 0 but smaller than 100. I've been getting ...
0
votes
0answers
5 views

What are different algorithms/methods of selecting triplet's for training a face recognition network?

I want to construct a siamese network using a triplet loss function. For which we need to select training samples( triplets ) for training the network, So how do we select hard triplet's for training ...
0
votes
0answers
71 views

Why does degradation occur in deep neural networks?

It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...
1
vote
0answers
17 views

How to determine which type of layer one should use?

Considering the SRGAN, I found it amazingly difficult to find logic on how this architecture was thought. The first two layers are input layer and 2d convolution layer, whose choice is pretty simple ...
0
votes
1answer
13 views

CNN for checking existance of single label

i am just thinking about training a neural network which uses data of only one single label. For example: Assuming i have many images which contain a dog. Now i want to teach the network how a dog ...
0
votes
2answers
55 views

How to learn certain Maths to understand machine Learning papers?

I have done the deeplearning.ai course on deep learning. But I cannot Understand equations like minGmaxDV(D,G)=Ex∼pdata(x)[logD(x)]+Ez∼pz(z)[log(1−D(G(z)))] ...
0
votes
1answer
20 views

Deep Neural Network and Activation Function

I want to know why we need Activation function in DNN hidden layers. I know a bit, like it will help us in, Increasing model complexity and introduce non-linearity Avoiding Gradient Vanishing ...
0
votes
0answers
34 views

Mathematical properties and deep learning

I have been exploring the application and behavior of deep neural networks in context of some mathematical problems. I am trying to observe which mathematical problems blend well and can be answered ...
0
votes
1answer
412 views

No gradients provided for any variable

I have composed a customized loss function (kl_loss): ...
2
votes
1answer
43 views

How to understand what each layer is learning in a Deep learning neural network?

In a recent answer I read on Stack Exchange, I read about a possible way to understand more clearly what happens in each hidden layer of a neural network. Here's the excerpt- You should watch ...
0
votes
1answer
232 views

What exactly is BatchNormalization() in keras?

A month or two straight away building image classifiers, I just sandwich the BatchNormalization layer between conv2d. I wonder, what it does, but I have seen my model learn faster in presence of these ...
1
vote
1answer
256 views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...
1
vote
2answers
421 views

clarify convLSTM usage for regression

I am trying to use keras and convLSTM layer to predict future weather data based on previous weather radar pictures. I use i timesteps i.e. i radar images as input data. In my diagram, I assume i=3. ...
1
vote
0answers
18 views

the difference meaning between latent and salient features

I'm learning deep model. the words latent and salient features usually repeat. I want to know the difference meaning between latent features and salient features
1
vote
1answer
34 views

Why is my MLP with 2 features is doing worse than MLP with 1 feature where the one feature is a combination of feature1*feature2?

I have programmed a MLP for a dataset (~500 rows) containing the length (L) and width (W) of an organism and the output of biomass (the organisms weight in pounds, B). ...
1
vote
0answers
140 views

Connection between piecewise linear basis functions and RELU activation function

ReLU activation is defined as follows $$\sigma(x)=\max(0, x).$$ Let's assume that I have deep network of 1 hidden layer, than output from my layer has form $$ f(x)= \sigma(Wx +b), $$ where matrix W ...
1
vote
0answers
29 views

Optimizing execution speed of a series of deep neural networks

I have a series of Neural Networks that I run on some video data. First network detects bounding boxes, then it extracts features for object tracking, matching each box to an id frame by frame and ...
0
votes
0answers
19 views

Can I have multiple (a sequence of) predicted values as the output of a neural network model?

I have a multi-dimensional time series data, and I want to use these data to do a time-series prediction. That is, the target(ground truth) of the training data is in a time series format instead of ...
2
votes
1answer
32 views

What is the difference? “Adding more LSTM layers” or “Increasing epochs”?

To get more accurate results which one is better? Adding more layers or increasing number of epochs? I like to know the difference between effects of these two approaches?
1
vote
0answers
24 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf it has written: " We present the first deep learning model to ...
2
votes
1answer
93 views

Are there real world applications where deep fully connected networks are better suited than ConvNets

I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets. I completed the first 3 courses in the deep learning ...
0
votes
1answer
137 views

What is the position embedding code?

https://github.com/google-research/bert/blob/master/modeling.py#L491-L520 The code of BERT is one of the implementation. But it is not what I need. I search a lot but can not judge. But where is ...
1
vote
0answers
46 views

Deep Learning Model apply for prediction real values [closed]

I have a dataset following structure: ...
1
vote
0answers
458 views

SSD based on ResNet-101 doesn't improve over SSD-VGGNet

I am training a SSD model for detecting mobile cranes. The training dataset contains 1000 images and test set over 400 images. About 200 epochs gave mAP 83%, but my target is 90%. So I trained SSD-...
3
votes
1answer
376 views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
1
vote
1answer
73 views

has number of output layer of DNN any effect in speed of find the optimal answer of DNN?

has number of output layer of DNN any effect in speed of find the optimal answer of DNN? For instance the more episodes is needed to train a DNN when the number of outputs is more? Is it correct?
4
votes
2answers
3k views

what is difference between the DDQN and DQN?

I think I did not understand what is the difference between DQN and DDQN in implementation. I understand that we change the traget network during the running of DDQN but I do not understand how it is ...
5
votes
2answers
3k views

Why should we use (or not) dropout on the input layer?

People generally avoid using dropout at the input layer itself. But wouldn't it be better to use it? Adding dropout (given that it's randomized it will probably end up acting like another regularizer)...
2
votes
1answer
343 views

Constant validation loss & accuracy, training accuracy fluctuates

I am training a Squeeze-net model for binary classification of images. I have 79968 images for training (50:50 for and against) and 8892 images in the validation set. After 35000 iterations my ...
5
votes
2answers
302 views

A Neural Network That Learns Bitwise XOR

I am trying to build a deep neural network that learns the coordinate-coordinate bitwise XOR of two matrices, but it performs poorly. For example, in the 2 bits case, its accuracy stays around 0.5. ...
3
votes
1answer
132 views

How to use a different model to deep neural network with reinforcement learning based on DQN?

Is it possible to implement a reinforcement learning algorithm without using a deep neural network (DNN) as used in deep reinforcement learning e.g. Deep Q-Network (DQN)? How can I replace the DNN in ...
3
votes
0answers
46 views

Deep Learning: Does starting the training on a smaller subset of the data make sense?

I trained a deep neural network with a small subset of my data, which allowed me to go through many epochs in a short amount of time and allowed the model to perform reasonably, then I gave it the ...
11
votes
2answers
377 views

Unable to figure out the linear embedding layer in the convolutional neural network?

I have the network architecture from the paper "learning fine-grained image similarity with deep ranking" and I am unable to figure out how the output from the three parallel network is merged using ...