Questions tagged [deep-learning]
a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.
4,840
questions
0
votes
0
answers
11
views
Using auxiliary softmaxes to measure impact of each submodule on the final softmax classifier
I am attempting to assess the impact of various submodules (CNN 1D, CNN 2D, CNN 3D, FFNN) on the final classifier of the neural network that i am currently building. The neural network itself is ...
- 101
0
votes
0
answers
17
views
Riemannian metric in Layer Normalization
I'm reading a paper about Layer normalization, and I couldn't find any clear explanation for this part:
Q1. Can anyone describe the derivation of the first equation in (8)?
Q2. I cannot understand ...
0
votes
0
answers
39
views
How to count number of circular pipes in an image using Deep Learning or Machine Learning?
I'm trying to build a program to count the number of pipes from a given image, here are some example test images.
[![enter image description here][2]][2]
[![enter image description here][3]][3]
I want ...
- 159
0
votes
1
answer
40
views
How did Andrej Karpathy make the LSTM output byte values for sampling Shakespeare?
I'm wondering how continuous output values of deep learning networks are converted to byte values or other discrete values for that sake.
For example here: In his famous article The Unreasonable ...
- 103
1
vote
0
answers
17
views
Can CNNs complete lines and contours?
Are there deep convolutional networks capable of recognizing two overlapping triangles in this image - or is this beyond the capabilities of CNNs?
And are there CNNs that can recogize two boxes ...
0
votes
1
answer
37
views
different range of target values in neural network
I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100).
The target ...
- 1
0
votes
0
answers
22
views
How to implement research paper network architecture algorithms
I'm struggling to implement this CNN architecture for this research paper: https://github.com/lindawangg/COVID-Net/blob/master/assets/COVIDNet_CXR.pdf. In Figure 2, the paper has a diagram that ...
- 85
0
votes
0
answers
13
views
Is a csv file to store image path and class neccessary for image classification?
I just get my hand-on a basic deep-learning project. I am working on multi-class image classification project with e-commerce dataset. I am not sure whether by storing training images in sub-folder ...
- 1
0
votes
0
answers
10
views
How should I save data in deeplearning with nosql (mongodb)?
I usually use file system to manage data for my deep learning model, but one of my boss told me to make nosql database to manage data.
Datasets I use have m rows, and n columns of count matrix and ...
0
votes
0
answers
20
views
How to generate synthetic data samples of Raman Spectroscopy by using GAN?
I am working on the Raman Spectroscopy dataset. The wavenumber/frequency range used by Raman spectra starts from 151.25 and ends at 1999.17. These values are used on the x-axis. While amplitude/...
0
votes
0
answers
18
views
Detection of noise and signal components with deep learning
There are thousands of datasets from the signal in the bellow image. With these datasets and machine learning, I want to first detect the jumping points (blue in the image) which are noises, and after ...
- 1
0
votes
0
answers
12
views
How to solve MemoryError problem?
I have audio signals which I need to convert them to melspectrogram. I am using Deepfilternet to remove background noise. when I use the output of Deepfilternet for next phase like padding, it showed ...
- 1
0
votes
0
answers
23
views
Python Darts API
I'm using the darts API for some time series prediction in python, while looking at the documentation for the RNN model I saw this parameters called training length. The description is not very clear ...
0
votes
0
answers
11
views
Why does weight decay produce regularisation in Deep Neural Network?
Weight decay penalizes the model to have smaller weights but how does this help in regularisation?
Any intuition on smaller weights => simpler model?
0
votes
0
answers
13
views
When training deep learning model which is better, training with sampled data Vs. training on shorter epoch
I am running multiple hyperparameter optimization trials therefore trying to find a way to reduce time consumption. Two ways that I could think of are
search hyperparameter on subset of data.
search ...
- 457
0
votes
0
answers
18
views
How would you treat imbalanced training data, and you don't know how test data distribution looks like in deep learning?
I posted this question on another place, but I want to get many tips, so I post here too.
I am building deep learning classification model in bioinformatics. I made training dataset by merging 12 ...
0
votes
0
answers
16
views
Sending rolling statistics to RNN
I'm curious if anyone has seen cases where sending rolling statistics such as mean, median, min, max, standard deviation, skewness, kurtosis, etc. have been helpful for model accuracy?
If so please ...
- 41
0
votes
0
answers
23
views
Different size of deep learning models but similar inference-time
I have three different semantic segmentation models with large differences in size. The first one includes 30,000,000 trainable parameters, the second one about 20,000,000 and the third one about 200,...
- 111
0
votes
1
answer
49
views
Shape of Flattened Layer in CNN
If I have a convolutional layer with dimension (5,5,4), (i.e, 4 no. of 5x5x1 feature maps), what will be the dimension of the flattened layer, if I apply flattening ...
0
votes
0
answers
31
views
Is it a good idea to use attention in VAEs for image generation?
There are research papers and codebases on GitHub that deal with VAEs for image generation on popular datasets like CelebA, etc. While surfing through Google Scholar I found self-attention and other ...
- 101
0
votes
1
answer
66
views
Label_fields are not valid error when using Albumentations
I'm not sure if you can have duplicates cross-forums, but my previous question on Stack Overflow was never answered. I'll paste it here just in case.
I'm using albumentations with the following code:
<...
0
votes
0
answers
13
views
How to analyze social media data to see its impact on a game's sales
I work for a console gaming giant. We forecasted the sales for a RPG game that was to be released few months back. But the actual sales was twice the forecast. This compelled the developers to ...
2
votes
1
answer
595
views
Input dimensions for the EfficientNetV2 family of models
I have a question regarding the EfficientNetV2 family of models. If my understanding is correct there are 6 models under this family - B0 to B1 & S are the comparatively smaller models while M &...
- 23
0
votes
0
answers
17
views
Decoder model selection for sequence-to-sequence ASR with XLS-R
I am new to building models for Speech ASR. I want to build a model for Speech Transcription in Urdu. I used XLS-R (following the example by Patric von Platen on Huggingface Colab notebook. My ...
- 23
0
votes
0
answers
22
views
What Can Prevent Time-Series Prediction Model From Learning Trend?
I am building an encoder-decoder prediction model based on this paper:
https://www.sciencedirect.com/science/article/pii/S0952197623001483
It is made of a transformer encoder and a 1D CNN Decoder. The ...
- 1
0
votes
0
answers
20
views
How to increase model's validation accuracy?
I'm trying to build a model for text similarity problem using CNN, Bi-directional GRU and Bi-directional LSTM. I've tried changing several parameters but I'm getting the maximum validation accuracy as ...
0
votes
1
answer
53
views
Are there any R packages that support Deep RNNs?
I recently found an interesting paper on what it really means for a recurrent neural network (RNN) to be deep here. Depth can be added in several different ways (state to state, input to state, etc.) ...
- 41
1
vote
1
answer
72
views
How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]
I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
0
votes
0
answers
12
views
Enumerable column in simple dataset
I have created my first dataset, which is divided into 3 smaller datasets. Main data is 1000 rows about cars, columns: model,year,mileage,price. Column "model" is enumerable (FiestaSedan,...
- 101
0
votes
1
answer
52
views
What neural network architecture would help me model a spectrogram?
I'm really a novice working with these technologies and I'm struggling to design a neural network that is powerful enough to model a spectrogram. For a personal project, I'm working on a spectrogram ...
- 3
0
votes
1
answer
34
views
Set a numerical range restriction on the output of a model
I am trying to train two models together. Both are regression models.
The first model would output a prediction, which is fed into the second model.
The second model is pre-trained to make good ...
0
votes
0
answers
40
views
Am I using a network that is too simple for the dataset/task?
I am training an RNN to classify some high-frequency financial data. A very good performance on this data would be an accuracy of >52% or so.
I have around 650K training examples and 150K dev set ...
- 103
0
votes
0
answers
14
views
Using neural networks to learn total distributions from linearly transformed samples
I am trying to explore the use of neural network based models to replace a cumbersome challenge with a faster more approximate method. The problem involves learning a distribution termed the "...
- 121
0
votes
1
answer
62
views
Use computer vision to detect door blockage
I want to detect door blockage on a camera. Basically if the exit door is blocked by an object, it detects it as an anomaly. How can we do it? Is it possible to do it using OpenCV?
Remember, it doesn’...
- 217
0
votes
0
answers
26
views
Why do I get such a low accuracy despite having good metrics?
I'm working on an image segmentation problem. I'm fine-tuning the network, and when displaying the metrics I obtain, I have some doubts.
I am going to provide some details about the network:
Solver: ...
- 304
0
votes
1
answer
50
views
I have created a CNN model and now i want to draw its architecture diagram can anyone help me with that
following is my architecture:
...
0
votes
2
answers
35
views
Is this an unusual distribution for a sigmoid output from a neural network?
Shown here is the histogram of around 130K predictions of my deep neural network that is classifying some financial data. This is on the dev set but a similar distribution is also seen on the train ...
- 103
0
votes
0
answers
115
views
Fine-Tune Llama on main and auxiliary task
I am trying to fine-tune Llama model on two task at the same time, using hugging face library:
Main task: Causal language model like the model was initially trained for
A classification task based on ...
- 11
0
votes
0
answers
69
views
Weight decay used by Adam optimizer for neural network caused NaN validation loss
I've built a model with BCE loss for CTR prediction in which the major part is a transformer encoder. I've used 0.1 for dropout probability. When using 0 weight decay for Adam the training and ...
- 392
0
votes
0
answers
27
views
Retraining a TFlite Model for Fall Detection on Smartphone Accelerometer Data
I have developed a CNN model for Fall Detection using Keras and converted it to a TFlite(TensorFlow Lite) model for integration into an Android app. The app allows users to collect samples, which can ...
0
votes
0
answers
21
views
How does the use of 1x1 convolutional layers represent permutation in the GLOW model?
I am currently reading the GLOW paper (found here) and I can not understand how the authors claim that the use of 1x1 convolutional layers is equivalent to permutation holds true.
I understand how a ...
- 1
0
votes
2
answers
45
views
Dropout and BatchNorm decrease speed of learning
Experimenting with the cifar10 dataset and faced with strange behavior when Dropout and BatchNorm don't help at all.
As I get:
Dropout - freezing some of the weights which helps us to prevent ...
- 103
0
votes
1
answer
32
views
Problem for a math formula in Weight Uncertainty in Neural Network
I am studying the paper https://arxiv.org/pdf/1505.05424.pdf and there is a formula I don't get page 4:
I don't understand how they obtain this formula. Moreover, with chain rule, I get $\frac{\...
- 1
0
votes
0
answers
47
views
BCE loss stuck at 0.693 in the beginnng of training and then started to decrease, why?
I'm using a Transformer encoder with a binary cross entropy loss for CTR prediction. The training batch loss is at around 0.693 constantly for the beginning several thousand steps (batches). I'm using ...
- 392
1
vote
0
answers
68
views
Annotating and Structuring a Dataset for Duplicate Detection
I'm currently working on a project that requires the detection of duplicate bands in Western blot images. The task involves two types of duplicates:
...
- 111
0
votes
0
answers
131
views
How to use pretrained encoder for customized Unet
if you have a standard Unet encoder such as resnet50, then it's easy to add pertaining to it. for example:
...
- 105
0
votes
0
answers
40
views
Tensorflow loss: 0.0000e+00 - accuracy: 0.0000e+00
I was making changes to improve myself in a chatbot code using LSTM. But Loss and truth values are getting ridiculous values.
Code:
...
- 1
0
votes
1
answer
22
views
Seeking guidance regarding course selection
I have completed Prof. Andrew Ng's Machine Learning Specialization a week ago and now exploring different competitions (Titanic, House Price Prediction, and from the Playground Series). I want to ...
2
votes
1
answer
34
views
Could resnet handles "one image" to "multiple ouputs" task?
I am not doing image classification. I need a model which take one image as input and ouput multiple values (all values could be larger than 1). Could resnet50 be used for this task?
I checked the ...
- 21
0
votes
0
answers
22
views
Overfitting on implemented Dense-Net architecture
I have been playing with different architectures and see how they would perform on the quick draw dataset. Even though the accuracy is significantly higher, I can't reduce overfitting no matter what I ...
- 1