Questions tagged [deep-network]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Data quantity is not low but data quality is low, what are the best practices now?

Text classification task, if data quantity is low but data quality is not low. We could use data augment methods for improvement. But the situation is that data quantity is not low and data quality ...
0
votes
0answers
13 views

Is this Tensorflow 1.x network get trained?

I've just started to learn Tensorflow 2.1.0 with Keras 2.3.1 and Python 3.7.7. I have found the following code from this "Omniglot Character Set Classification Using Prototypical Network": ...
1
vote
0answers
9 views

Detecting adversarial examples: Building an adversary detection network

I'm trying to understand how an adversary detection network would work. In section 3.2 of this research paper - I've found some sort of an explanation as to how it might be structured + how the ...
0
votes
1answer
29 views

Implementing the SVHN CNN architecture in Srivastava et al. 2014 Dropout paper

I am trying the implement the CNN architecture introduced in Srivastava et al. 2014 Dropout paper (appendix B.2), for the SVHN dataset. I implemented only the convolutional layers part, without ...
2
votes
1answer
36 views

How to prevent vanishing gradient or exploding gradient?

Whats causing the vanishing gradient or exploding gradient to occur, and what are the measures to be taken to prevent it?
1
vote
0answers
50 views

Understanding depthwise convolution vs convolution with group parameters in pytorch

So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows. For a input feature map of (C_in, F_in, F_in), we take only 1 ...
2
votes
0answers
80 views

How to detect vanishing and exploding gradients with Tensorboard?

I have two "sub-questions" 1) How can I detect vanishing or exploding gradients with Tensorboard, given the fact that currently write_grads=True is deprecated in the Tensorboard callback as per "un-...
1
vote
0answers
27 views

Post-classification after inference in deep learning models

I designed a fire detection using Deep Learning binary classification in Keras (fire vs none). It's a simple model with a few layers. In my training dataset, I ...
1
vote
1answer
45 views

How does ResNet bottleneck architecture's input size is possible to change from 56x56x64 to 56x56x356?

In ResNet papaer, First residual block's input size is 56x56x64 caused by 7x7x64 filter in first layer. But, in the paper, they showed residual block that has 56x56x256 input size. How does it is ...
1
vote
0answers
93 views

Deep reinforcement learning with multi-dimensional action

I am trying to design reinforcement learning algorithm. My action and state space are continuous. Action, which I would like to take can be represented by a matrix, lets say of dimension $n \times n$. ...
2
votes
1answer
16 views

Does it make sense to expand word embeddings so that each array index is a feature input or should the embedding itself be a model input?

If you are building a DNN, say, with two layers, and you want to use embeddings as one of your feature inputs, what's the best way to input the embedding? I'm trying to understand if I should break ...
4
votes
1answer
38 views

What are some (non image) datasets where deep networks are required for good performance?

It is well known that deep networks tend to outperform shallow networks and other classical machine learning techniques such as boosting on learning tasks involving images. I believe this is because ...
0
votes
0answers
109 views

How to calculate the memory usage of a deep LSTM network?

I was trying to estimate the memory usage for my LSTM network by referring to an examples of CNN memory usage calculation at http://cs231n.github.io/convolutional-networks/#computational-...
0
votes
1answer
42 views

Apply LSTM to each matrix element with Keras

I'm trying to apply a LSTM/GRU to each entry of a matrix $X$ note: Each matrix element is a time-series, so shape of X is (batch_size, rows, cols, time_steps, dims) $ y_{i,j}= \begin{cases} ...
0
votes
0answers
12 views

How to change classification model architecture for a new target application

I'm new to Deep Learning with Keras. With some tutorials online for cat vs non-cat classification, I was able to compile this simple architecture for my own ...
1
vote
1answer
114 views

Learning rate Scheduler

A very important aspect in deep learning is the learning rate. Can someone tell me, how to initialize the lr and how to choose ...
0
votes
0answers
42 views

NL2SQL task, if we have enough data, what will the model achieve for hard SQL?

We are afraid that the hard SQL like TABLE JOIN is the limit for industrial application. Addition info: https://yale-lily.github.io/spider Thank you very much.
0
votes
0answers
20 views

What will happen if we replace the transformer of BERT to evolved transformer?

If we replace the official BERT's transformer to evolved transformer, do the change accelerate the inference speed without losing accuracy?
1
vote
2answers
33 views

When would not normalizing input values have higher accuracy?

Right now I'm training a deep neural network for a binary classification problem, with a feature set of winrates. As such, each winrate is bigger or equal to 0 but smaller than 100. I've been getting ...
0
votes
0answers
10 views

What are different algorithms/methods of selecting triplet's for training a face recognition network?

I want to construct a siamese network using a triplet loss function. For which we need to select training samples( triplets ) for training the network, So how do we select hard triplet's for training ...
0
votes
0answers
200 views

Why does degradation occur in deep neural networks?

It has been shown that "plain" neural networks tend to have an increased amount training error, and accompanied test error, as more layers are added. I am not quite certain as to why this occurs. In ...
1
vote
0answers
18 views

How to determine which type of layer one should use?

Considering the SRGAN, I found it amazingly difficult to find logic on how this architecture was thought. The first two layers are input layer and 2d convolution layer, whose choice is pretty simple ...
0
votes
1answer
34 views

CNN for checking existance of single label

i am just thinking about training a neural network which uses data of only one single label. For example: Assuming i have many images which contain a dog. Now i want to teach the network how a dog ...
0
votes
2answers
68 views

How to learn certain Maths to understand machine Learning papers?

I have done the deeplearning.ai course on deep learning. But I cannot Understand equations like minGmaxDV(D,G)=Ex∼pdata(x)[logD(x)]+Ez∼pz(z)[log(1−D(G(z)))] ...
0
votes
1answer
22 views

Deep Neural Network and Activation Function

I want to know why we need Activation function in DNN hidden layers. I know a bit, like it will help us in, Increasing model complexity and introduce non-linearity Avoiding Gradient Vanishing ...
0
votes
0answers
40 views

Mathematical properties and deep learning

I have been exploring the application and behavior of deep neural networks in context of some mathematical problems. I am trying to observe which mathematical problems blend well and can be answered ...
0
votes
1answer
2k views

No gradients provided for any variable

I have composed a customized loss function (kl_loss): ...
2
votes
1answer
75 views

How to understand what each layer is learning in a Deep learning neural network?

In a recent answer I read on Stack Exchange, I read about a possible way to understand more clearly what happens in each hidden layer of a neural network. Here's the excerpt- You should watch ...
0
votes
1answer
893 views

What exactly is BatchNormalization() in keras?

A month or two straight away building image classifiers, I just sandwich the BatchNormalization layer between conv2d. I wonder, what it does, but I have seen my model learn faster in presence of these ...
1
vote
1answer
485 views

Difference between Dueling DQN and Double DQN?

I have read some articles, but still can not figure out the difference between the Dueling DQN and Double DQN? What exactly is the difference between them? Also, Does Dueling DQN need to be built on ...
1
vote
3answers
777 views

clarify convLSTM usage for regression

I am trying to use keras and convLSTM layer to predict future weather data based on previous weather radar pictures. I use i timesteps i.e. i radar images as input data. In my diagram, I assume i=3. ...
1
vote
0answers
22 views

the difference meaning between latent and salient features

I'm learning deep model. the words latent and salient features usually repeat. I want to know the difference meaning between latent features and salient features
1
vote
1answer
42 views

Why is my MLP with 2 features is doing worse than MLP with 1 feature where the one feature is a combination of feature1*feature2?

I have programmed a MLP for a dataset (~500 rows) containing the length (L) and width (W) of an organism and the output of biomass (the organisms weight in pounds, B). ...
1
vote
0answers
167 views

Connection between piecewise linear basis functions and RELU activation function

ReLU activation is defined as follows $$\sigma(x)=\max(0, x).$$ Let's assume that I have deep network of 1 hidden layer, than output from my layer has form $$ f(x)= \sigma(Wx +b), $$ where matrix W ...
1
vote
0answers
40 views

Optimizing execution speed of a series of deep neural networks

I have a series of Neural Networks that I run on some video data. First network detects bounding boxes, then it extracts features for object tracking, matching each box to an id frame by frame and ...
2
votes
1answer
54 views

What is the difference? “Adding more LSTM layers” or “Increasing epochs”?

To get more accurate results which one is better? Adding more layers or increasing number of epochs? I like to know the difference between effects of these two approaches?
1
vote
0answers
26 views

What is the meaning of the Variant Q-learning and To what INPUT and OUTPUT refer? in Abstract of DeepMind DQN paper 2013

-INPUT and OUTPUT OF ATARI DQN: In the abstract paragraph of the DQN work by DeepMind https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf it has written: " We present the first deep learning model to ...
2
votes
1answer
175 views

Are there real world applications where deep fully connected networks are better suited than ConvNets

I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets. I completed the first 3 courses in the deep learning ...
0
votes
1answer
211 views

What is the position embedding code?

https://github.com/google-research/bert/blob/master/modeling.py#L491-L520 The code of BERT is one of the implementation. But it is not what I need. I search a lot but can not judge. But where is ...
1
vote
0answers
49 views

Deep Learning Model apply for prediction real values [closed]

I have a dataset following structure: ...
1
vote
0answers
523 views

SSD based on ResNet-101 doesn't improve over SSD-VGGNet

I am training a SSD model for detecting mobile cranes. The training dataset contains 1000 images and test set over 400 images. About 200 epochs gave mAP 83%, but my target is 90%. So I trained SSD-...
3
votes
1answer
782 views

What are the effects of clipping the reward in stability?

I am looking for stabilizing my results of DQN, I found clipping is one technique to do it but I did not understand it completely! 1- what are the effects of clipping the reward, clipping the ...
1
vote
1answer
84 views

has number of output layer of DNN any effect in speed of find the optimal answer of DNN?

has number of output layer of DNN any effect in speed of find the optimal answer of DNN? For instance the more episodes is needed to train a DNN when the number of outputs is more? Is it correct?
5
votes
2answers
5k views

what is difference between the DDQN and DQN?

I think I did not understand what is the difference between DQN and DDQN in implementation. I understand that we change the traget network during the running of DDQN but I do not understand how it is ...
6
votes
2answers
4k views

Why should we use (or not) dropout on the input layer?

People generally avoid using dropout at the input layer itself. But wouldn't it be better to use it? Adding dropout (given that it's randomized it will probably end up acting like another regularizer)...
2
votes
1answer
466 views

Constant validation loss & accuracy, training accuracy fluctuates

I am training a Squeeze-net model for binary classification of images. I have 79968 images for training (50:50 for and against) and 8892 images in the validation set. After 35000 iterations my ...
5
votes
2answers
374 views

A Neural Network That Learns Bitwise XOR

I am trying to build a deep neural network that learns the coordinate-coordinate bitwise XOR of two matrices, but it performs poorly. For example, in the 2 bits case, its accuracy stays around 0.5. ...
3
votes
1answer
176 views

How to use a different model to deep neural network with reinforcement learning based on DQN?

Is it possible to implement a reinforcement learning algorithm without using a deep neural network (DNN) as used in deep reinforcement learning e.g. Deep Q-Network (DQN)? How can I replace the DNN in ...
3
votes
0answers
72 views

Deep Learning: Does starting the training on a smaller subset of the data make sense?

I trained a deep neural network with a small subset of my data, which allowed me to go through many epochs in a short amount of time and allowed the model to perform reasonably, then I gave it the ...
11
votes
2answers
450 views

Unable to figure out the linear embedding layer in the convolutional neural network?

I have the network architecture from the paper "learning fine-grained image similarity with deep ranking" and I am unable to figure out how the output from the three parallel network is merged using ...