Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

What's intuition behind the activity regularizer? Any practical application?

I have read and understand that the activity regularizer is to operate on a neural layer output to make it smaller. However, I couldn't have any intuition for it and don't know why making output ...
0
votes
0answers
11 views

How to change classification model architecture for a new target application

I'm new to Deep Learning with Keras. With some tutorials online for cat vs non-cat classification, I was able to compile this simple architecture for my own ...
0
votes
0answers
8 views

back propagation through time derivation issue

I read several posts about BPTT for RNN, but I am actually a bit confused about one step in the derivation. Given $$h_t=f(b+Wh_{t-1}+Ux_t)$$ when we compute $\frac{\partial h_t}{\partial W}$, does ...
0
votes
0answers
14 views

Is using cross-entropy enough to ensure the output is a distribution probability?

I am following along https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html. In this code, the last layers of the pretrained networks are linear. The loss used in this ...
0
votes
0answers
11 views

DNN loss gets smaller but accuracy stays the same

I am learning a DeepNN to choose between three decisions in a simulation. Therefore, I can run the simulation as often as I want and can generate as many samples as I want. Based on this tutorial (...
0
votes
0answers
5 views

if two convolution layer connected in tandem follow associative property of convolution?

Two Convolution filter follow the associative property as follows :- I want to ask whether this property will hold for two convolution layer with no operation in between them?
0
votes
0answers
10 views

Conceptual help generating a text adventure game using a GAN

I have built a playable dungeon crawler game that lets a character progress through a series of randomly generated rooms filled with doors, chests, stairs, etc. Ideally, I would be able to display a ...
0
votes
0answers
12 views

Need of value vector in transformers

I am reading the paper "Attention is all you need" (https://arxiv.org/abs/1706.03762). In transformer architecture, we have 3 vectors(key,value and query) for each word. I don't understand the need of ...
2
votes
1answer
117 views

NLP Transformers: How to get a fixed sentences embedding vectors size?

I'm loading a language model from torch hub (CamemBERT a French RoBERTa-based model) and using it do embed some sentences: ...
0
votes
2answers
33 views

Fine Tuning the Neural Nets

I have recently read about Fine Tuning, and what I want to know is, when we are fine-tuning our model is it necessary to Freeze the model and train only the top part of the model and then unfreeze ...
0
votes
0answers
19 views

Actor Critic Model implementation

I am going to work on a project which requires implementation of A2C model using Tensorflow 2.0. I am new in the Machine Learning field and also in Python. These are topics which I have covered ...
0
votes
1answer
14 views

TensorFlow: CosineDifference ObjFunc Constant throughout training

The following example is a simplified version of what I'm working on. I'm trying to find a neural network which minimises the cosine distance. The reason I have implemented my own cosine difference ...
4
votes
1answer
35 views

Is it possible to generate syllogisms using an NLP algorithm?

I want to build a tool that generates sensible syllogisms. An example of a syllogisms is: all A are B. all C are A. all C are B). I want the triplet (A, B, C) to be semantically related to each other ...
1
vote
1answer
8 views

Tools suitable for semi-automatic video labeling?

So far I have been using labelme to label objects in videos I use for training, but it is quite time consuming. Are there good tools to help with that? I was thinking about a tool where I label some ...
0
votes
0answers
15 views

Inference speed of CNN Model is varying time to time, why?

I have a light CNN classification model with not more than 131,000 parameters. When I try to run the model and check the inference speed, it is changing from time to time (14 to 17 fps). My laptop ...
2
votes
1answer
41 views

Time Series Generation - Multi Dimensional Time Series Data

Disclaimer: Mathematicians please don't be mad at me for the use of some of the terminologies in this post. I am an Engineer. :-) Background: So I am currently working on a problem where I have to ...
2
votes
0answers
105 views

What are the tradeoffs between Bayesian Deep Learning and Deep Gaussain Processes?

I understand the differences between Deep Gaussian Processes(DGPs) and Bayesian Deep Learning(BDL): DGPs are essentially feed-forward neural networks where each node is a Gaussian Processes, which BDL ...
0
votes
1answer
15 views

Why the first prediction of neural network in PyTorch is slower than following predictions?

So I have ResNet50 trained to classify images. For each prediction I track the time needed for it (input and model are moved to GPU): ...
0
votes
0answers
16 views

Which is best FaceNet or dlib_face_recognition_resnet_model_v1?

I am newbie in face recognition related things... As far i observed dlib's frontal_face_detectoris widely used to find the faces in an image and after that, to ...
0
votes
1answer
18 views

How to build in symmetry of inputs into a Deep Neural Network?

I have a Deep Neural Network that takes $n$ inputs $X = [X_1, \ldots, X_n]^T$ and gives $n$ ouputs $Y = [Y_1, \ldots, Y_n]^T$. Normally, I can just do a standard deep neural network with a few fully ...
1
vote
2answers
25 views

Unsupervised image classification?

Does this exist? What algorithm or combinations of algorithms would be able to classify images without supervision? For example if you have many pictures of cats and dogs, then without being trained ...
0
votes
1answer
27 views

Detect if my ANN model is overfitted

I've been trying the kaggle dataset of Credit card fraud detection Dataset . I've used ANN using keras and tensorflow. You can find the code in the screenshot. The only problem is im getting accuracy ...
0
votes
1answer
23 views

How to convert photo to a vector drawing

I am using the following code to convert photo to a drawing: ...
1
vote
0answers
19 views

Generative Adversarial Networks - The simplest possible examples

I'm looking for the simplest possible examples of GANs. What would be simple yet illustrative examples with, say, univariate data and in which both the generator and the discriminator are as simple as ...
0
votes
0answers
72 views

Adapting Pytorch tutorial “NMT from Scratch…” for dynamic RNN

I have taken the code from the tutorial and attempted to modify it to include bi-directionality and any arbitrary numbers of layers for GRU. Link to the tutorial which uses uni-directional, single ...
1
vote
1answer
84 views

AlphaGo Zero loss function

As far as I understood from the AlphaGo Zero system: During the self-play part, the MCTS algorithm stores a tuple ($s$, $\pi$, $z$) where $s$ is the state, $\pi$ is the distribution probability over ...
0
votes
0answers
5 views

Do I need different CNN architectures to detect the same objects for the same dataset with higher fps and higher resolution?

I am planning to do object detection on a dataset that is 5 fps with a resolution of 720 x 320. After training that CNN on that dataset, how significantly should I modify the CNN architecture to ...
0
votes
0answers
14 views

What is “style normalization” referred to in the Adaptive Instance Normalization paper?

In "Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization", the authors argue that the significant performance boost from instance normalization is not only due to contrast ...
0
votes
0answers
6 views

BatchNorm vs InstNorm from the perspective of feature distributions

What I understand so far... The main purpose of BatchNorm is to overcome covariance shift -- more specifically what the authors of BatchNorm coined "internal covariance shift". Covariance shift is ...
1
vote
1answer
43 views

Learning rate Scheduler

A very important aspect in deep learning is the learning rate. Can someone tell me, how to initialize the lr and how to choose ...
2
votes
0answers
16 views

Different convolutions in CNN

I have a simple question. Why only convolution is used in CNN? There are a lot of possible rules for combining a filter and an image. Why is pixel-wise convolution the standard? For example, dropout ...
1
vote
1answer
21 views

Large amount of Sigmoid outputs are ones and zeros

I have Keras neural network for binary classification with final layer having one output with Sigmoid activation. I have noticed that large amount of output numbers are strictly one or zero (rather ...
0
votes
0answers
6 views

Research on explaining generalizability of deep learning methods

I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite ...
1
vote
1answer
174 views

How to draw neural network diagrams with this particular style?

I would like to draw a neural network architecture with the follow style. Do you know which tool can be used to do this? The paper is Operation-aware Neural Networks for User Response Prediction.
1
vote
2answers
284 views

what is the difference between euclidean distance and RMSE?

I'm searching for a loss function that fits my Project. Actually I have two question but they are in the same direction. I take a look at the definition of the root mean squared error and the ...
1
vote
1answer
15 views

How would keras model learn at every epoch?

I know concept of Epochs, batch size and iteration. let's say, Total_data = 6400 Batch_size = 64 Iteration = 100 In this, basically we are taking in 64 data points to computer memory and ...
0
votes
0answers
6 views

How bit precision while training can impact DNN's accuracy - Libraries that would support inference with quantized types

I would like to check how bit precision impacts DNN's accuracy. Do you know any C/C++/Python libraries that wouldn't require huge rework for supporting inference with quantized types? For example, I ...
1
vote
0answers
21 views

How can I increase the speed and performance of my implementation of an AI for Reversi?

I made an AI for Reversi, aka Othello (8×8), like Alpha Zero, using this book. This book is written in Japanese. The source code of the AI I implemented can be found in this Github repository. There ...
1
vote
1answer
24 views

Can the same CNN architecture be used for different data sets?

I have a CNN architecture that works well on 32x32x3 images. Can I use that same architecture for a data set made up of 28x28x1 images? (Both data sets have 10 classes). If this is possible, what ...
0
votes
0answers
15 views

Mixing unsupervised and supervised algorithms in image classification model

I am trying to replicate the general image classification model used in a paper that I cite later below. The following image is an extract from a paper that proposes a novel method of performing image ...
2
votes
1answer
32 views

Why in this case are gradient steps not perpendicular to contour lines?

There is a theorem that gradient at point is perpendicular to tangent line to contour line at given point. Why in this picture it seems that this rule is not respected? source: http://www....
-1
votes
2answers
42 views

Activation Functions in Neural network

I have a set of questions related to the usage of various activation functions used in neural networks. I would highly appreciate if someone could give explanatory answers. Why is ReLU is used only ...
0
votes
0answers
36 views

Running LSTM model on a big data sample using pyspark

I was wondering how does one run an LSTM model on a big dataframe in pyspark. Ideally, one wants to run the model parallelly on different nodes of a spark cluster. But how does one do that?
0
votes
0answers
34 views

Accuracy of CNN on images taken under different conditions

I have a dataset containing images taken under 4 different conditions. When training the model, I use the same proportion of images (25%) from each condition. Then, I'm testing on 4 different test ...
0
votes
0answers
39 views

Combining 2D Detection with Disparity Maps to Learn 3D Object Geometry

Since the disparity map above is a representation of the object's distance from the camera's origin, is it reasonable to assume that a network (perhaps a convolutional LSTM) could be trained to ...
0
votes
0answers
16 views

VGG style transfer fails with pre-computed `vgg19.preprocessing()` for the content, transfer, style inputs

I'm working through the style transfer tutorial on tensorflow, see: style transfer I made a few adjustments to my notebook, but it works fine for the base case: [ content_image | transfer_image | ...
0
votes
1answer
29 views

Deep Learning for Video Classification

Which Deep Learning architecture is best for classifying short videos of variable length? I would like to classify videos that last from 1 up to 3 seconds.
0
votes
0answers
91 views

Error loading a model (.h5 file) after training yolo-keras classification model

I am working on realtime object detection using my laptop's camera with Yolo and Keras. I have trained a model and the resulting output is a .h5 file containing (from my understanding) the model and ...
1
vote
1answer
88 views

How to calculate accuracy, precision and recall, and F1 score for a keras sequential model?

I want to calculate accuracy, precision and recall, and F1 score for multi-class classification problem. I am using these lines of code mentioned below. ...
0
votes
0answers
16 views

saving a model during training of an RL agent

I am training an RL agent using PPO2 algorithm. Iam using stable-baselines library. During the training process, my rewards are slowly increasing and stabilizing, but are falling down suddenly. I ...